Opinion: Scientific Arguments Against Creative Testing

When Steve Jobs was asked if Apple had used research to help design and launch the iPad, he replied “no”. When asked why not, he explained, “it’s not the consumer’s job to know what they want”.

 

A better answer would be, “the consumer isn’t very good at telling you what they want”. Yet most marketing research is based on asking them what they think about products, brands or ideas, whether it does this by questioning them in focus groups or having them answer questions in quantitative research.

 

So if you have been arguing in vain against creative testing, Draftfcb’s seasonal gift to you is a list of irrefutable scientific objections to it. 

 

1. Research makes people pay more attention than they do in the real world (and that’s not necessarily a good thing).

 

Robert Heath, author of ‘The Hidden Power of Advertising’ wrote that we tend to be in one of two modes when we are processing information. One is ‘high involvement processing’, when we are actively paying attention to something. The other, based on our attention running in the background, is low involvement processing. 

 

These modes work in quite different ways. High involvement processing enables us to remember logical detail and recall limited amounts of it very accurately, but only for a short period of time after paying attention. These memories are triggered voluntarily. Low Involvement Processing, on the other hand, seems to seed memories that are triggered by external episodes where there is some association. These are often more emotional, last longer and seem more powerful in terms of evoking action. 

 

2. People are in denial about their irrationality.

 

Dan Ariely dissects this in his book ‘Predictably Irrational’. We hate to see ourselves as irrational, which is why respondents will often choose a more rational approach or one that is easily rationalised. But in fact, the reasons we give for decisions that we’ve made are not very often our real motivations. Most rational reasons are post-rationalisations of emotional or instinctual decision making processes. Even scientists who study this find it hard to explain their own behaviour in the heat of the moment. Yet billions of pounds are spent on research that asks respondents to do exactly that.

 

3. If people think they are going to have to explain a choice, it affects what they choose.

 

In an experiment by Timothy Wilson and Douglas Lisle, two cells of respondents were asked to choose a poster from a range that spanned representative (two puppies playing) to abstract (modern art). Both cells were told they could return their poster if they didn’t like it. Before they made their choice, however, one cell was told they would be asked to explain why they liked the poster they chose. 

 

Two very interesting things happened. The first: there was a notable difference in the types of posters chosen by each cell. On average, the cell that didn’t have to explain themselves chose more abstract posters. The cell that did have to do so chose posters that were more representative. The need to explain led people to make choices they could explain – two puppies could be explained as something simple, like ‘I used to have dogs as a kid’. Whereas a print of a Modigliani? Much more difficult.

 

The really interesting finding was when the respondents were contacted three weeks later. After being asked if they were happy with their posters, the respondents who had been asked to give their reasons were significantly less satisfied with their choice. 

 

According to the researchers: “When people think about reasons, they appear to focus on attributes of the stimulus that are easy to verbalise and seem like plausible reasons, but may not be important causes of their initial evaluations.” 

 

4. What people say they like can change.

 

What people say in a research setting is dictated by their mental and physical state at the time. Neuroscience shows that when people are hungry, they respond differently to images of food than when they aren’t. A bland research room with office furniture may be more stressful than your couch at home, but less stressful than the weekend shop. Behavioural research shows our levels of stress affect how we respond to information.

 

But it doesn’t stop there. Sheena Iyengar, author of ‘The Art of Choosing’ refers to an experiment where people were shown pictures of two attractive women – one blonde, one brunette. 

 

They were shown a whole set of different pairs of female pictures, and asked which ones they thought were prettier. They were then shown the pictures they chose again and asked why they picked them. In some cases though, unbeknownst to the respondent, the images were switched – if they had chosen the brunette they were sometimes shown the blonde. What did they do? 87 per cent of the time they didn’t even notice. They simply said, “Oh, I prefer blondes”… even though they had actually chosen the brunette!

 

Happy Christmas!

 

Matthew Willcox, director of Draftfcb's Institute of Decision Making