SurveyMonkey Logo
Plans & Pricing

Product testing: How to conduct an effective product assessment

Get your product concepts and ideas validated by a trusted audience—in less than an hour.

According to research, around 30,000 new products are launched every year, and of those 95% end up failing.

All too often, in a bid to recoup investment or to beat off the competition,  brands rush new products to market with testing whether it resonates with consumers, or indeed whether the product is something that people actually want. But, even the most experienced decision-makers struggle to make the right calls without input from their target audience.

Whether you’re looking for the safest bet or wondering if a risky concept might actually pay off, product testing can help you find a concept that meets your goals and resonates with your audience. With a product concept test, you can gauge customer sentiment and find the proven winners in a pool of different product ideas.

Product testing gives your target audience a range of options and asks them to provide feedback on each one. With product testing, you can compare metrics like purchase intent, quality, and value to find out which product concepts customers like best.

In this article, we’ll take a closer look at why you should test your product concepts, how to conduct an effective product assessment, and what you can do to get accurate feedback from your respondents.

Be confident about your product before launch
Get your concepts vetted by your target audience first.
See how

Product testing

Product testing gives your target audience a range of options and asks them to provide feedback on each one. With product testing, you can compare metrics like purchase intent, quality, and value to find out which product concepts customers like best. Drawing on those insights, you can craft products that will resonate, sell and engender loyal customers. Let’s take a closer look at what product testing entails, and why you need to do it. 

What is product testing?

Sometimes described as consumer testing or comparative testing, product testing is the process of measuring the properties or likely performance of products among a target audience. While a great product can recover from a less than stellar launch, an impressive launch can’t save a product that nobody wants to buy. By asking for audience feedback on your product concepts before moving forward with a launch, you can identify successful product ideas right off the bat. This ensures that  you commit money and resources to the right projects.

What are the goals of product testing?

You can use product testing to reach a number of different goals:

Determine whether a new product development program is on track

If you’re developing a new product, one of the worst things you can do is to wait until the end of the development stage to test whether your product lands with those people you want to market it to. The most successful products are tested regularly throughout the development process in order to demonstrate proof of concept and to make sure that the final product is exactly what the customer wants.

Decide suitability for end use

Product marketing doesn’t end when the customer leaves the store or gets the product to their home. If you want people to come back time and time again, or to shout about the product to prospective new customers, you’ll need to validate the suitability of your product for the end user. As a bonus, if you test your product with different segments of consumers, you’ll not only learn more about how the product works, but how different end categories of customers use it.

Introduce your product to the customer

One of the biggest causes of product failure is the failure to adequately communicate the purpose of the product to the market. Product testing can help ensure that consumers understand what your products will do for them, and which products offer them the best value.

Solve problems with current products

Too often, new products are recalled or have to be developed because of problems only recognized once they’re sold. Product testing can help to avoid costly errors or oversights by having the end user try out the product and send that all important feedback.

Increase consumer protection

If you’re distributing a new product or service, it's in your best interest to check the claims made during marketing campaigns. So, before you advertise your product as the cheapest, fastest or best in class, test it with your target audience

Identify potential cost savings

The cost of developing or redeveloping products can be tremendous. Oftentimes, features are added that customers don’t really value, or tweaks are made that make little difference to the bottom line. By testing your product before it reaches the market, you can make sure that every penny of your research and development adds value.

Why you should test your product concepts before launch

When you use product testing, you take on a product launch with 4 major advantages:

  1. Get buy-in on your product concept: People can’t argue against hard numbers. Testing proves which products resonate most with your target audience, which helps you pick clear winners and persuade other stakeholders to get behind those choices.
  1. Gain confidence in features that help you stand out and learn why others are lagging: Maybe your audience liked one product best, but thought a simpler variation of that product was more useful. Or maybe they loved a product, but had concerns about privacy. You can use this information to combine the best features of each option and bundle them all into one great product.
  1. Know by which demographic your product concept resonates: By filtering your responses, you can see how different groups (age, gender, location, etc.) feel about your product concepts. This information helps you match your product with an ideal market.
  1. Build agility into your workflows: By frequently collecting data through agile market research—surveys sent to a sample of your target consumers on a regular basis—your business can make informed decisions without relying on data from third-party insights organizations. This helps you react more quickly to changes in your market.
What is your product experience like?
Momentive, the makers of SurveyMonkey, offers solutions to help improve your product experience—from concept testing to product optimization.
Learn more

How to put together a successful product assessment

Now that you know why it’s important to test your product concepts before settling on one, you’re ready to put together your test survey. For accurate and actionable results, follow these 4 steps:

Each of the 4 steps for testing any product.

Choose product concepts to test

While you can test any product concept you want, it’s best to choose stimuli that you’re already somewhat confident in based on an informal survey of coworkers and friends. To make sure your survey results are useful, choose products in a similar stage of development. If you’re testing a finished product against a product you’re still working on, the winner is obvious.

Survey design options

To make the testing process more manageable for you and your respondents, limit the number of stimuli in your survey. The maximum number of stimuli you should include in your test depends on whether you plan to use a monadic survey design or a sequential monadic survey design.

Monadic survey design

A monadic survey design divides up your respondent pool and presents each respondent with a survey asking for feedback on a single stimulus. When you’ve gathered enough feedback on each stimulus, you can pick a winning concept by aggregating the feedback from all your respondents.

So, if you have a sample size of 500 respondents, you ask 250 of them about one product option and 250 of them about the other.

Here’s an example of a product testing survey with a monadic design.

This survey design lets you ask more questions about each stimulus, which lets you figure out more about how  individual features resonate, and which shortcomings might prove difficult in the future. It’s also more likely to result in a relatively short questionnaire, which benefits your survey’s quality of data and completion rate. However, since you’re only showing each respondent one stimulus, you’ll need to target a larger audience. This can prove expensive and it might not be as easy to execute.

Sequential monadic survey design 

A sequential monadic survey design presents every respondent with the same survey that asks for feedback on multiple stimuli. Once you’ve collected responses from a statistically significant number of respondents, you can pick a winning concept by looking at the most common reactions to the stimuli.

In this type of survey, if you have a sample size of 500, you’d ask all 500 about both product ideas. 

Here’s what your product testing survey might look like if you use a sequential monadic design.

This type of design lets you target a smaller audience than a monadic design, which makes it more cost-effective. However, if you want to keep your survey to a manageable length, you won’t be able to ask as many questions about each stimulus—meaning fewer insights about the specifics of each product.

Learn more about the pros and cons of each design, and how you can set them up in SurveyMonkey.

Decide which product assessment metrics are ideal

What makes a good product? The metrics you choose to measure will help you decide.

The questions you ask in your product testing survey can be as specific as you’d like. If you want to know whether your audience finds a certain feature useful, for example, you might ask about the innovativeness, relevance, and value of that feature. In general, these are some key metrics you might want to include in your survey:

  • Appeal: Is your product enticing to potential customers?
  • Believability: Is your product’s messaging believable?
  • Innovativeness: Is your product innovative?
  • Purchase intent: Do people want to buy your product?
  • Quality: Does your product seem high quality?
  • Relevance: Does your product fulfill your audience’s needs and wants?
  • Uniqueness: Is your product different from other products?
  • Value: Is your product a good value?

The relative value of each metric you measure depends on your goals for the product. If you want a product that will sell well, purchase intent may be the most important metric for you. If your main goal is to win loyal customers, you’ll likely place more importance on quality.

We suggest using the Likert scale to test your metrics. This scale is simple and intuitive for respondents, offers consistent choices, and makes it easy for you to analyze results. If you choose this scale, your questions can follow the formula, “How (metric) is the product?” where your answer choices range from “Extremely (metric)” to “Not at all (metric).”

For example, here’s how the question might look if your metric is innovativeness:

How innovative is this product?

  1. Extremely innovative 
  2. Very innovative
  3. Somewhat innovative
  4. Not so innovative
  5. Not at all innovative

Sometimes, it isn’t clear from a respondent’s answers which concept they like best. Resolve this issue by including a forced-answer question that asks respondents to choose their favorite product from all the options at the end of the survey. The results of this question will also help you rank the importance of each metric to the overall likeability of your product. For example, if respondents consistently choose the product with the highest value as their favorite, you can assume value is a key value driver for your target market.

You might also want to include screener, category, and demographic questions in your survey. These questions tell you more about the people you’re collecting feedback from and let you segment your results by group. 

Have your target audience evaluate the options

Your target audience can help you decide which product concepts are worth pursuing and which you should set aside. There are 2 ways to get feedback from this group of people:

  • A focus group lets you collect in-depth feedback on a variety of different questions. They can be costly to organize, however, and you might not be able to gather feedback from a representative sample of your target audience.
  • SurveyMonkey Audience is a market research panel that lets you target who you want to reach and then collect feedback from them in minutes.Audience is more cost-effective than a focus group, but a little less in-depth.

Find a winner with analytics

Once you’ve collected your survey responses, you’re ready to compare your product concepts. You can use Top 2 Box scores to make your data clearer and your findings more pronounced. It groups together positive and negative responses to each question into a single percentage, which is easier to compare.

You also need to look at open-ended responses. Our word cloud feature can make them easier to grock.

A word cloud from a particular product.

How product testing fits into the product life cycle

Product testing belongs to the development and growth stages of the product life cycle, when you’re testing new product ideas and adding new inventory.

A chart that displays the different steps of the product life cycle and the tests to run during each of them.

During the development stage, you’ll probably be looking for broad feedback that’ll help you make major decisions about your products. This is the time to ask respondents to choose between several very different product concepts, for example. Because everything about your brand is new at this point, it’s the best time to field broad, directional feedback on your products.

During the growth stage, you may be looking for more nuanced feedback on your product concepts. This is the time to ask respondents focused questions about new product features or niche alternatives to current products, for example. It’s also the time to think about sampling your loyal customers to see if they’re on board with your new ideas.

Tips for accurate product testing

By following a set of best practices for product testing, you can get better feedback from your respondents. Here are 5 things to keep in mind:

  1. Keep the metrics constant across stimuli. Ask the same set of questions about each product concept, so you can accurately compare each option.
  2. Provide high-resolution images of your product renderings or prototypes. If respondents can’t see the product well, they might not be able to give accurate feedback.
  3. Get feedback from a statistically significant sample pool. Learn how to determine if your sample pool is large enough. To make sure you reach enough people in your target audience, think about using our panel.
  4. Limit the number of questions you ask in your survey. We recommend asking no more than 30 questions per product concept test. More questions increase the chance that respondents put less thought into their responses or drop out of your survey completely.
  5. Benchmark your results with existing products. If you’re building on an existing product, include that product in your survey. The results will help you decide whether or not your new concept can compete with your original product or the products of your competition.

Since you understand why testing your products is important, know how to run your own test, and have some useful testing tricks up your sleeve. 
Now it’s time to jump into your own product testing survey. If you want results fast, we can get you some: in less than an hour.

Run quick and effective concept testing with SurveyMonkey expert solutions