SurveyMonkey Logo
Plans & Pricing

Product testing: How to conduct an effective product assessment

95% of product launches fail—often because a brand pushes them to launch without stopping to find out if the product is something people actually want. Even the most experienced decision-makers struggle to make the right calls without input from their target audience.

Whether you’re looking for the safest bet or wondering if a risky concept might actually pay off, product testing can help you find a concept that meets your goals and resonates with your audience. With a product concept test, you can gauge customer sentiment and find the proven winners in a pool of different product ideas.

Product testing gives your target audience a range of options and asks them to provide feedback on each one. With product testing, you can compare metrics like purchase intent, quality, and value to find out which product concepts customers like best.

Let’s take a closer look at why you should test your product concepts, how to conduct an effective product assessment, and what you can do to get accurate feedback from your respondents.

Ready to start testing product ideas?
Master every step that’s involved by reading our ultimate guide to concept testing.
Get guide

Why you should test your product concepts before launch

While a great product can recover from a less than stellar launch, an impressive launch can’t save a product that nobody wants to buy. By asking for audience feedback on your product concepts before moving forward with a launch, you can identify successful product ideas right off the bat. This ensures that  you commit money and resources to the right projects.

When you use product testing, you take on a product launch with 4 major advantages:

  1. It’s easier to convince others to back your product concept. People can’t argue against hard numbers. Testing proves which products resonate most with your target audience, which helps you pick clear winners and persuade other stakeholders to get behind those choices.
  2. You know which features (or potential features) are great—and which would need improvement. Maybe your audience liked one product best, but thought a simpler variation of that product was more useful. Or maybe they loved a product, but had concerns about privacy.  You can use this information to combine the best features of each option and bundle them all into one great product.
  3. You know where your product concept resonates. By filtering your responses, you can see how different groups (age, gender, location, etc.) feel about your product concepts. This information helps you match your product with an ideal market.
  4. Frequent testing helps you react to change quickly. By frequently collecting data through agile market research—surveys sent to a sample of your target consumers on a regular basis—your business can make informed decisions without relying on data from third-party insights organizations. This helps you react more quickly to changes in your market.

How to put together a successful product assessment

Now that you know why it’s important to test your product concepts before settling on one, you’re ready to put together your test survey. For accurate and actionable results, follow these 4 steps:

Each of the 4 steps for testing any product.

Choose product concepts to test

While you can test any product concept you want, it’s best to choose stimuli that you’re already somewhat confident in based on an informal survey of coworkers and friends. To make sure your survey results are useful, choose products in a similar stage of development. If you’re testing a finished product against a product you’re still working on, the winner is obvious.

To make the testing process more manageable for you and your respondents, limit the number of stimuli in your survey. The maximum number of stimuli you should include in your test depends on whether you plan to use a monadic survey design or a sequential monadic survey design.

A monadic survey design divides up your respondent pool and presents each respondent with a survey asking for feedback on a single stimulus. When you’ve gathered enough feedback on each stimulus, you can pick a winning concept by aggregating the feedback from all your respondents.

So, if you have a sample size of 500 respondents, you ask 250 of them about one product option and 250 of them about the other.

This survey design lets you ask more questions about each stimulus, which lets you figure out more about how  individual features resonate, and which shortcomings might prove difficult in the future. It’s also more likely to result in a relatively short questionnaire, which benefits your survey’s quality of data and completion rate. However, since you’re only showing each respondent one stimulus, you’ll need to target a larger audience. This can prove expensive and it might not be as easy to execute.

A sequential monadic survey design presents every respondent with the same survey that asks for feedback on multiple stimuli. Once you’ve collected responses from a statistically significant number of respondents, you can pick a winning concept by looking at the most common reactions to the stimuli.

In this type of survey, if you have a sample size of 500, you’d ask all 500 about both product ideas. 

Here’s what your product testing survey might look like if you use a sequential monadic design.

This type of design lets you target a smaller audience than a monadic design, which makes it more cost-effective. However, if you want to keep your survey to a manageable length, you won’t be able to ask as many questions about each stimulus—meaning fewer insights about the specifics of each product.

Learn more about the pros and cons of each design, and how you can set them up in SurveyMonkey.

Decide which metrics you want to look at

What makes a good product? The metrics you choose to measure will help you decide.

The questions you ask in your product testing survey can be as specific as you’d like. If you want to know whether your audience finds a certain feature useful, for example, you might ask about the innovativeness, relevance, and value of that feature. In general, these are some key metrics you might want to include in your survey:

  • Appeal: Is your product enticing to potential customers?
  • Believability: Is your product’s messaging believable?
  • Innovativeness: Is your product innovative?
  • Purchase intent: Do people want to buy your product?
  • Quality: Does your product seem high quality?
  • Relevance: Does your product fulfill your audience’s needs and wants?
  • Uniqueness: Is your product different from other products?
  • Value: Is your product a good value?

The relative value of each metric you measure depends on your goals for the product. If you want a product that will sell well, purchase intent may be the most important metric for you. If your main goal is to win loyal customers, you’ll likely place more importance on quality.

We suggest using the Likert scale to test your metrics. This scale is simple and intuitive for respondents, offers consistent choices, and makes it easy for you to analyze results. If you choose this scale, your questions can follow the formula, “How (metric) is the product?” where your answer choices range from “Extremely (metric)” to “Not at all (metric).”

For example, here’s how the question might look if your metric is innovativeness:

How innovative is this product?

  1. Extremely innovative
  2. Very innovative
  3. Somewhat innovative
  4. Not so innovative
  5. Not at all innovative

Sometimes, it isn’t clear from a respondent’s answers which concept they like best. Resolve this issue by including a forced-answer question that asks respondents to choose their favorite product from all the options at the end of the survey. The results of this question will also help you rank the importance of each metric to the overall likeability of your product. For example, if respondents consistently choose the product with the highest value as their favorite, you can assume value is a key value driver for your target market.

You might also want to include screener, category, and demographic questions in your survey. These questions tell you more about the people you’re collecting feedback from and let you segment your results by group. 

Want help from market research pros? Looking for ways to refine your skills?

These resources might be just what you’re looking for!

Ultimate guide to running market research: This resource has everything you need to run market research, from planning your study to taking action!

Product testing survey template: This expert-certified survey template can help you brainstorm your questions. You can also use it and edit it however you’d like.

SurveyMonkey Audience: Our global consumer panel allows you to survey people in more than 100 countries. 

Market Research Services: From survey consultation to team trainings, our experts will ensure that your research on SurveyMonkey Audience runs smoothly.

Have your target audience evaluate the options

Your target audience can help you decide which product concepts are worth pursuing and which you should set aside. There are 2 ways to get feedback from this group of people:

  • A focus group lets you collect in-depth feedback on a variety of different questions. They can be costly to organize, however, and you might not be able to gather feedback from a representative sample of your target audience.
  • SurveyMonkey Audience is a market research panel that lets you target who you want to reach and then collect feedback from them in minutes.Audience is more cost-effective than a focus group, but a little less in-depth.

Find a winner with analytics

Once you’ve collected your survey responses, you’re ready to compare your product concepts. You can use Top 2 Box scores to make your data clearer and your findings more pronounced. It groups together positive and negative responses to each question into a single percentage, which is easier to compare.

You also need to look at open-ended responses. Our word cloud feature can make them easier to grock.

A word cloud from a particular product.

How product testing fits into the product life cycle

Product testing belongs to the development and growth stages of the product life cycle, when you’re testing new product ideas and adding new inventory.

Every phase of the product life cycle.

During the development stage, you’ll probably be looking for broad feedback that’ll help you make major decisions about your products. This is the time to ask respondents to choose between several very different product concepts, for example. Because everything about your brand is new at this point, it’s the best time to field broad, directional feedback on your products.

During the growth stage, you may be looking for more nuanced feedback on your product concepts. This is the time to ask respondents focused questions about new product features or niche alternatives to current products, for example. It’s also the time to think about sampling your loyal customers to see if they’re on board with your new ideas.

Product testing doesn't have to take a lot of work
Our Product Concept Analysis solution can help you build your survey, field it, and analyze its responses.
Learn more

Tips for accurate product testing

By following a set of best practices for product testing, you can get better feedback from your respondents. Here are 5 things to keep in mind:

  1. Keep the metrics constant across stimuli. Ask the same set of questions about each product concept, so you can accurately compare each option.
  2. Provide high-resolution images of your product renderings or prototypes. If respondents can’t see the product well, they might not be able to give accurate feedback.
  3. Get feedback from a statistically significant sample pool. Learn how to determine if your sample pool is large enough. To make sure you reach enough people in your target audience, think about using our panel.
  4. Limit the number of questions you ask in your survey. We recommend asking no more than 30 questions per product concept test. More questions increase the chance that respondents put less thought into their responses or drop out of your survey completely.
  5. Benchmark your results with existing products. If you’re building on an existing product, include that product in your survey. The results will help you decide whether or not your new concept can compete with your original product or the products of your competition.

That’s it! Now you understand why testing your products is important, know how to run your own test, and have some useful testing tricks up your sleeve. Now it’s time to jump into your own product testing survey.

See how SurveyMonkey can power your curiosity