Putting Your Survey to the Test: Final Checklist

How to ensure the quality of your survey before you send it.

matrix_questionYou’ve set a goal for your survey, finalized your questionnaire, determined how you’re going to collect your responses, but are you ready to send your survey to respondents?

But before you do, make sure your survey is in tip-top shape for public consumption by testing it out. Here’s a checklist with ten things to look for before you hit send:

1. Preview and test your survey

The most important thing to do before sending out your survey is to preview it. When testing your survey out, keep these things in mind:

  • Time how long it takes you to finish. Keep in mind that your respondents will perceive the time to finish as much longer than you will. If it seems too long, it is. You might consider cutting out some questions or adding skip logic to streamline the experience.
  • Preview the survey on a mobile device. Surveys look different and completion times vary depending on whether you are on a computer, tablet, or smartphone. Surveys even look different on Android and iOS smartphones, so you may need to change the format of your questions if you think that a lot of your respondents will be completing the survey on a smartphone.
  • Get feedback from a friend or colleague who didn’t work on the survey. They can give you a sense of whether it is too long and also catch any mistakes that you may have missed.

2. Write an introduction

An introduction is a great way to let your respondents know what the topic of your survey is and also thank them for their time, so don’t forget to include one. A simple sentence will do. At a loss what for what to write? Just borrow it from your survey goal! Try something like:

“Your feedback on our iPhone app will help us improve it and fix any problems. Thank you for taking the time to complete our survey.”

And if you have already have an introduction, make sure it isn’t too long. A long, text-heavy intro can turn off respondents, and they may be less likely complete the survey.

Free Survey Writing eGuide
Get tips from our survey experts on how to write great surveys that’ll get you reliable results.
Get the Guide →

3. Read the survey through the eyes of the respondent

Survey creators often agonize over the content in their survey and are so focused on the questionnaire that they often forget to make the survey relatable to their respondents. So put yourself in the shoes of your respondent and be on the lookout for questions that:

  •    Use jargon (vocabulary that you are familiar with, but your respondents may not be).
  •    Use complicated language. Instead, try simpler language that is easily interpreted by everyone.
  •    Are double barreled and ask about two concepts in the same question.
  •    Respondents are not qualified (or don’t know how) to answer.

4. Double check your answer options

Are you providing all possible answer options to your questions, or should you add a “Not applicable” option? By not offering all possible answer options, your respondents may feel like they are forced to randomly pick an answer option to your question, giving you bad data.

Or are you using scales? If so, make sure you are consistently using the same number of scale points throughout your whole survey. If one of your scales only has four answer options, but the others have five, you may be losing an opportunity to compare that question with others.

5. Check for typos

Make sure to check for any spelling or grammatical mistakes. An easy way to do this is by reading the survey aloud to yourself!

6. Add pages

Give your respondents a mental breather by adding in pages to your survey. Often survey creators have ten or more questions on a page and respondents don’t have time to give their brains a rest, making survey fatigue much more likely. Having too many questions on a page can also make surveys harder to take for respondents on mobile devices.

But there is a fine line here. Each question doesn’t need to be on its own page (unless you are using skip logic), and having too many pages can require a lot of load time.

7. Review skip logic

Check all possible paths of your skip logic! Mistakes in the programming can skip respondents to the wrong page, which will harm your data. We’ve even seen logic errors that send respondents in an infinite loop so there is no way “out” of the survey.

If you haven’t used skip logic, think about whether it makes sense for everyone who responds to your survey to answer every single question. For questions don’t apply to everyone, add in skip logic.

8. Randomization

Randomization is good way to get rid of order bias. Think about whether your questions would benefit from either having the answer options randomized, or if the questions themselves should be randomized.

If you do use randomization, make sure to double check that it is working properly. Sometimes survey creators forget to anchor the last answer option (by not clicking “Don’t randomize last choice”, which means that the “None of the above” option doesn’t show up at the bottom and instead is randomly placed in the list), or think that they randomized something when they actually didn’t.

9. Email content and subject line

If you are emailing out a link to your survey or using our email collector, make sure that the content of your email is appropriate. This is often the first time that you interact with your survey respondent, so make a good impression by writing a short email that has the following information: a sentence about the topic of the survey, when you’d like them to complete the survey by, and of course, a link to the survey (or information on how they can complete it if you aren’t sending a link).

You can also think about using your introduction sentence in the body of the email instead of in your survey. Also, try sending out a few test emails to make sure that the content of your email or the subject line doesn’t get caught in anyone’s SPAM filters.

10. Pat yourself on the back!

You did it, you’ve got a great survey. Now congratulate yourself, send it out to your respondents, and watch their answers roll in!

So before you send out your next survey, make sure you’ve gone through this checklist and completed all the ten items on this list! Once you have and made your final edits, you’re ready to start collecting data!

This article is part of SurveyMonkey’s Surveys 101 project. We hope to help more people create smart surveys. Learn more about the project and our involvement in the research community.

Get the Feedback eGuide