Beware Creative Testing

I’ll start off by admitting that our healthcare marketing team at Jennings tests creative concepts on a fairly regular basis; so we are not opposed to creative testing. But we believe that one should proceed with caution when venturing into this territory. I’ll give you a real world example to demonstrate just how difficult it is to come to know the mind of the consumer.

With this entry is a graphic of two charts showing results from creative testing we did in the Raleigh/Durham area of North Carolina more than a year ago. We tested twelve TV spots from four area hospitals (Duke, UNC, WakeMed and Rex Healthcare); and 10 print ads from the same group of hospitals.

As you’ll see from the chart, when we pooled the data from the two focus groups (20 participants) the difference between the most popular TV spot and the least popular was a 5.85 rating vs. a 4.35 rating. That’s on a scale of 1 to 10 – with 10 being “like it very much.” So the most preferred spot only received a 5.85 on a scale of 10!

With print ads it was the same story. The least preferred TV spot received a 3.9 and the most preferred received a 5.6. Again, on a scale of 10. This points out the obvious challenge: If you go into creative testing expecting a slam dunk for your campaign, you will be disappointed if you haven’t first adjusted everyone’s expectations for what a slam dunk looks like. Consumers are fickle and tastes vary. You’re never going to please everyone – especially when you are trying to create something that is memorable and that breaks through the clutter.

Of course, the real fly in the ointment is that people don’t always remember the advertisers who are featured in the ads they report “liking.” I’m not convinced there is always a correlation between likability and memorability.

A more important point is that what people “like” changes over repeated exposure to a new concept. Think about new car models that are released. As an example, I can remember disliking the Honda Element when it was first released. But it has grown on me over time and now I really like it. The car didn’t change; my preference changed after repeated exposure to the vehicle. Often it takes us time to grow to like something new; and that includes TV commercials and ad campaigns. Again, this points out one of the dangers of creative testing. All you are testing is an initial reaction at a given point in time. There is very little predictive value.

It stands to reason that in advertising you can’t please everybody in the target audience with a given creative execution. My fear is that if you try to satisfy all tastes with advertising, you’ll water it down to the point where it won’t be memorable or engaging. From my perspective, if you can engage half the audience with a given ad or TV spot, you’ve hit a homerun.

When it comes to creative testing methodology, my preference is using one-to-one interviews with consumers, rather than focus groups. The group dynamic is simply not as important in creative testing. I’d much rather capture an individual’s gut reaction to an ad. I want to know what the advertising says to them about the advertiser and whether or not they find it relevant to their life. When using focus groups, we have participants record their reactions on answer sheets, and we delay all conversation until we’ve reviewed all of the creative. One of the risks with the focus group format, if you allow participants to talk as you go, is that the participants learn as you review the first few pieces of creative and their “evaluation” of the advertising becomes informed by the taste and judgments of other participants who were vocal early on in the process. My advice is to keep everyone silent until they have committed their reactions to paper, and then go back and discuss how they felt about individual pieces of creative. And certainly encourage dissenting opinions.

How hard is it to make an impact on the consumer? Really hard! If you would like to review some interesting and disturbing data, just last week a Nielson study was released that reported the following:

“Of the nearly 1,000 consumers Nielsen has interviewed to date, only a third could recall any TV commercials they had seen. And it’s not just advertising that TV viewers apparently have trouble remembering. Amazingly, 21% of TV viewers could not correctly recall at least one TV program they had viewed.” (Source: Media Posts, Media Daily News, August 8, 2007)

Our work is cut out for us. If you’ve got any questions about creative testing, feel free to shoot me an email at ddunlop@jenningsco.com.

0 comments on “Beware Creative Testing

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: