Survey Science
How to NOT write your online survey questions
How to NOT write your online survey questions

Spring break is around the corner and your just-barely-in-college teenager mentions that she’s thinking of going to Florida. Without you. You, being the supporting parent that you are, blurt out something like:

“Do you want to go to Florida and make me worry about you the whole time? Yes or no?”

“Well, no,” the kid (young woman?) succinctly tells you, but that doesn’t really tell you as much as you’d hoped.  When she responds, “no,” she might actually mean a variety of things…

  1. I very much would like to go to Florida, but I wouldn’t like you to worry.
  2. I actually don’t want to go to Florida because I hate the sunshine, but your worry is irrelevant.
  3. The whole Florida idea is actually kind of dumb and I didn’t really want to go, but I like when you worry, it makes me feel special!

And so on. As you can see, a simple yes or no answer doesn’t really tell you everything you need to know.

Questions with two parts embedded in them (like trips to Florida and parental worry) are known as “double-barreled” questions. Although double-barreled questions may be helpful rhetorical devices for guilt tripping your kids (hey, we’re not judging)—they do not belong in surveys.

Were this uncomfortable parental conversation a survey instead, it would be better to have two questions prepared:

  1. Do you want to go to Florida? (yes/no)
  2. Do you want me to worry about you the whole time? (yes/no)

Now, as a parent you might not be ready to hear the answers to those individual questions, but as a researcher trying to understand what people are thinking—you need to hear the truth.

Want even more richness? Instead of asking your two questions as yes/no questions, add the stem of “how much”:

  1. How much do you want to go to Florida?
  2. How much do you want me to worry about you the whole time?

Need help labeling response answers? Check out our blog post.

Want even more richness? Ask another “open-ended” question of “Why?” as a follow-up to your first question. And then analyze the results with our cool word cloud tool.

Just remember, save the double-barreled questions for parenting. Not for surveys.