Ask more, know more, do more

Managing Survey Response Quality with Category and Method Exclusions

Managing Survey Response Quality with Category and Method Exclusions

The TrueSample team is continually exploring the nuances of survey data quality that impact the results of online market research.  Our goal is to continue to ask questions and explore the possibilities as we guide the industry in assuring the highest possible data quality in quantitative market research.

Today, we’ll look at category and method exclusions – these are often discussed as good avenues for managing survey response quality.  In the context of these discussions, a “category” refers to a type of product or service (for example, skin care products) and a “method” refers to a type of study being conducted (like a concept test).

Many in the market research industry are interested in category and method exclusions because there is a belief that, in some cases, presenting the same respondents with several similar surveys in a short period of time may cause them to respond differently than they otherwise would.  This could be due to boredom, an increased ability to game the survey, or other reasons. But the results are the same—an inability to make valid apples-to-apples comparisons across respondents, and a lack of trust in the resulting data.  As if that weren’t bad enough, sending respondents too many surveys that seem too much alike may eventually stop them from participating in market research altogether.

So how do we avoid such problems?  First, we need to track the appropriate categories and methods for each survey.  Next, we need to be able to tie together that survey’s respondents and the date they entered the survey with the categories and methods we’re tracking.  That’s all the information that we need to make the exclusions – we would be able to answer a request like, “Give me a sample of women in their 20s, but exclude anyone who took a survey about makeup in the last 30 days.”

Unfortunately, the solutions that exist today are often quite fragmented and time consuming, requiring researchers to use multiple tools and multiple steps to make category and method exclusions.  In some cases, researchers are able to make these exclusions only with previous respondents to their own surveys.   To overcome these hurdles, companies with a commitment to data quality (like TrueSample) are working on developing technology solutions that will make this effort fast, simple and with less fragmentation across the industry.

What hurdles have you faced in your own survey creation and research? Let us know if the comments section below.

Tags: , , , ,

One thought on “Managing Survey Response Quality with Category and Method Exclusions

  1. I ‘m happy to see your blog , some very meaningful and helpful knowledge .Thanks a lot for sharing.

Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help


Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide