“Short cuts make long delays.”
― J.R.R. Tolkien, The Fellowship of the Ring
Shortcuts definitely have their appeal. But when you use them in surveys it’s a disaster waiting to happen. Here are three of the most common shortcuts we see in surveys, why they’re bad, and how to fix them up right!
1. Writing a “kitchen sink” survey
The excuse: Don’t know what to ask? Just ask a little bit of everything.
The problem: If you don’t know why you’re asking the questions, your respondent definitely won’t know why either. Long surveys that don’t really hang together as a coherent conversation are frustrating and confusing for respondents. And when respondents get frustrated and confused they’re more likely to dropout, less likely to be paying close attention to your questions, more likely to be answering questions randomly for spite, and so on. This is bad for you because it will take you longer to finish collecting data for your survey and the data you get will be worse. Also, asking about too many different concepts means that you don’t get to explore any of them in depth. This means you pass up your opportunity to gain helpful insights into what’s really going on.
The solution: Pick a goal, any goal. But only pick one. As we’ve mentioned before, having a research question is incredibly important. It’s also important that you only have one per survey. Have more than one thing you’re looking to understand? Make more than one survey.
2. Writing a “kitchen sink” question
The excuse: Don’t know what to ask? Just keep it vague and general.
The problem: Similar to writing a kitchen sink survey, people like to ask really general questions like, “Are you satisfied with this service?” The problem with this is that no one knows what you mean, and everyone who answers it thinks of something different. This introduces a lot of random error into your data and means that one person could be thinking about the price of your service, another about the quality, another about the convenience, and another just quits in frustration because he doesn’t know what you’re talking about. It also hampers your ability to make decisions based on the data you get back. If someone is “dissatisfied” with your service, what do you change to fix it?
The solution: Avoid this mess by keeping questions specific and focused. As we’ve mentioned before, staying construct specific will keep both you and your respondents focused and your data quality high.
3. Writing a “lazy” question
Here are some question types to use with extreme caution:
The excuse: These question types are all used because they make your life easy as the survey creator. Simply put, they make the process of survey making way less painful for you.
The problem: Instead of YOU putting in the hard work, these question types force your respondents to put in the hard work. Not only is that kind of mean and unfair of you, your respondents will rebel. The bottom line? They will get lazy and answer in ways that don’t reflect their TRUE attitudes. And this is bad.
The solution: Check out the links above for specific fixes, but as an overall rule of thumb, use a unipolar Likert scale. Why? This question type talks to respondents in a language they understand, keeping them focused. Learn more here.
So don’t get sloppy, survey-makers and start taking shortcuts. Put in the hard work up front so that it’s smooth sailing for your respondents. As an added bonus, extra planning up front will save you valuable time on the back end because your analysis phase will be a snap.
Thoughts? Let us know below!