Concept testing is a very valid approach to explore appeal of product and services and is highly recommended when first deciding if a product is worth your time and money to develop.
But when these test are not well designed, this type of research tends to yield general information that is not very actionable or worst yet, could be misleading. Creating a concept test correctly is key to making informed decisions.
The first step in concept testing is to create product concepts that clearly define the “reasons-to-believe” in the product and the experience the customer may expect from it.
The best source for good product concept descriptions is the potential customer.
Conducting qualitative market research as the first step in the concept testing process can bring a lot of value by uncovering what is important to customers and how customers talk about the product.
I recently was invited to take an online survey which was used to conduct a concept test to measure the appeal of online marketing research courses. I think back now and wonder if the sponsor of the survey did his homework in trying to identify what features really matter to potential end-users of online marketing research courses.
The image below is a snapshot of how the concept was presented in the online survey. Of the four features described in the concept, the only one I found relevant is the 24/7 accessibility. The other three are quite vague and somewhat immaterial.
What does it matter if the courses will have 3 to 8 hours of total instruction if I don’t know the length of each class? Is it 30 minutes or 60 minutes?
My work schedule and home life may only allow me to sit for 20 minutes at a time to attend an online class.
Furthermore, I don’t know what they mean by “comprehensive instructions.”
Are the classes task-oriented, will I need to do a lot in preparation to take the class? I associate the word “instructions” with manuals, so when I read this I feel exhausted already. Who reads manuals these days?
“Dynamic interfaces” and the image they use don’t tell me much either.
A more concrete example would have helped to set my expectations. Without any direction, my expectations could be too high, which can lead to deep disappointment if the courses don’t align with my notions of what a dynamic interface is in this context.
A short video showing different screens would have provided a better illustration of what the user can expect from the interface.
Finally, the most relevant feature to me is the course topic and this is absent from the product concept.
One may contend that there are many topics of interest and there is interest in testing only the format. On the hand, I would argue that the format is irrelevant if the topic is not of interest. The approach followed here doesn’t really reflect how the end-user thinks about this product.
If topic is the most important feature when selecting an online class, then excluding it creates an artificial product concept to which it is difficult to give an accurate answer. When they asked if I would be interested in these courses, my only plausible answer was “might or might not.”
I have a feeling that that study produced misleading results for them, if I had so much trouble with filling out the answers. That was an example of what not to do.
So what are the key steps in conducting product concept tests that yield accurate results?
- Conduct qualitative research with members of the target audience. Avoid “inbred” concepts based only on input from internal staff – A few interviews with marketing research professionals would have provided a richer and more realistic foundation on which to base the concept.
- Create concepts that are concrete and put them in relevant context – An online course on a particular topic would have allowed me to give a more realistic response. A video could have shown me what to expect from the “dynamic interface.”
- Write concepts in the target audience’s language – No “instructions” please!
- Focus on Reasons-to-Believe arguments that highlight what is important to your audience – I need more than 24/7 accessibility to assess the value of this service. Topics are a good start.
If you’d like more help with concept testing, please head over to Relevant Insights’ and see our Concept Testing and Product Optimization services.
