How to get the most out of concept test analysis

Tapping into an unmet market? That’s like catnip for marketers and one of the most effective ways to do that is through concept test analysis.

So it shouldn’t be any surprise for those of you out there in marketing land to hear a ton of talk and discussion about concept testing.

But how do marketers make the most out of this type of analysis? Is knowing the winning concept good enough to take the plunge, or do we need to investigate more to actually be confident in potential concepts? Are marketers missing something here?

Without testing the concepts, marketers always risk losing out on a great idea. But if the concept elements do not leverage the actual need insight effectively, it might make sense to revisit the concepts and test them again. Ideas based on strong insights have a high probability of success, even with average marketing support.

And be sure to look at the right ones. Concepts can be evaluated simply based on purchase intent. Although high purchase intent scores can indicate a potential winning concept, other metrics could also add significant insight into “true” purchase intent.

For example, Value for Money (VFM) can be a crucial metric—VFM can give you even more information on whether price sensitivity may actually influence consumer buying behaviors or not. Similarly, Purchase Frequency can be important for repeat-purchase categories, especially if a concept is a consumer-packaged good. At the end of the day, the metrics you choose should directly relate to the type of conceptual product you are trying to assess.

When it comes to concept testing, there are few possible benchmarking options. While benchmarking amongst new concepts in a test is the most valid and reliable measure to compare concept responses from the same sample, it doesn’t necessarily indicate what to expect in terms of in-market performance.

In situations where in-market performance is a key decision criteria, benchmarking to similar products in the market can also prove beneficial. Benchmarking against the norm from a previous concept test is also known to be effective, provided the data is strong and relevant to the new set of concepts being tested

After deciding on metrics and benchmarks, the final step is to create the action rules that specify which actions will be taken when high and low scores are observed. Decisions about confidence intervals (typically 90% or 95%) indicate how marketers want to manage the risk of interpreting an observed difference as a real difference.

Preferences for looking at higher confidence intervals are not ideal for every situation. For example, in the case of a re-stage product concept, looking at a higher interval among users make sense. However, looking at lower intervals among non-users ensures marketers aren’t ignoring any possible advantage that the restaged concepts may have amongst this group of consumers.