Ask more, know more, do more

Quality Online Survey Design Yields Quality Research Results

Quality Online Survey Design Yields Quality Research Results

Experienced researchers have long assumed that online survey design, respondent engagement, and online research data quality are interrelated.  For example, it seems obvious that long, complex questionnaires will likely be viewed unfavorably by respondents, and will increase the likelihood of “bad” respondent behaviors such as speeding and partial completes.  Along the same lines, it is reasonable to suppose that sub-optimal respondent behavior and experience may expose potential issues with the quality of data in these surveys.  For these reasons, it is commonly accepted that the primary objective of a well-designed survey is to elicit the most accurate and representative possible responses from the respondents.

Recent studies conducted by the TrueSample team clearly show that survey design influences a respondent’s experience in a survey, which manifests itself in the way they behave while answering survey questions – and that the level of respondent engagement directly impacts the quality of respondent data.  If respondent engagement is low, data quality is also more likely to be low.  Additionally, complex survey design can cause respondents to become unengaged and increase the odds of unreliable data.

How can you tell if an online survey is well-designed?

TrueSample has developed tools that provide a comprehensive, objective measure of survey engagement to help researchers see the impact that their survey design has on respondent engagement.  TrueSample SurveyScore® uses a proprietary mathematical formula for looking holistically at respondent engagement – using both quantitative behavioral metrics (such as survey abandonment and speeding) and qualitative experiential metrics (such as respondents’ ratings of the survey-taking experience) in a given survey.

  • Survey Rating:  how the respondent perceived the survey experience, self-reported based on a standard scale.
  • Partial Rate:  the percentage of respondents who abandoned the survey before completing it.
  • Speeding:  the level of speeding behavior exhibited by the survey respondents.

Together, these metrics comprise a SurveyScore, which is assigned to every TrueSample-enabled survey. The SurveyScore also provides a way to consistently evaluate respondent engagement across surveys, in varying product categories, and across different research methods.  You can compare the results of two surveys and know that the one with the lower SurveyScore probably has lower data quality.

Can you check survey design quality before launching a survey?

TrueSample SurveyScore Predictor helps researchers optimize survey design before it is launched, allowing them to estimate and improve respondent engagement for an online survey.  This tool calculates an estimated SurveyScore based on survey design elements such as types of questions asked and complexity of questions – and then will suggest which particular elements should change to improve your SurveyScore and lead to better engagement levels.

In addition to keeping “good” respondents engaged, good survey design also has the additional benefits of lowering field times, reducing costs and, given the finite supply of available online panelists, helping to maintain online panel health throughout the industry.

We believe that survey engagement measurement tools such as TrueSample SurveyScore are critical for researchers making business decisions based upon their survey results.  By measuring and benchmarking the respondent engagement level of each survey before and after the survey is launched, researchers can improve survey design to increase respondent engagement and enhance data quality.

For more information on the TrueSample respondent engagement study, see the article reprint from Quirk’s Marketing Research Review: Designed to engage: What is the impact of survey design on respondent engagement?

Tags: , , , , ,

One thought on “Quality Online Survey Design Yields Quality Research Results

  1. it’s important to check the design of your survey to make it be attractive

Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help


Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide