Ask more, know more, do more

A Little Bias is a Big Problem

A Little Bias is a Big Problem

Susan and Walter are getting ready for their big summer barbecue party. Susan agrees to be the barbecue grill master if Walter buys all the ingredients. Not sure how many hamburgers to buy, Susan sends out a survey to all of the guests. When Walter checks the survey, he’s shocked. 0% of people want hamburgers?! Then he reads Susan’s survey question:

Do you want to have someone kill a defenseless animal, skin it, grab some of it, add preservatives to it, and force me to inhale its death fumes while I cry silently because it reminds me of all my animal farm friends from when I was a child?

Walter shakes his head and sighs, “Do I have to do everything myself?” He writes a new survey question and sends it out to their friends:

Do you want to support local American farmers in these troubled economic times by grilling up a traditional American juicy burger?

“There,” he thinks, “that’s better.”

So who’s right? Susan or Walter?

Actually, they’re both wrong. Both Susan and Walter have created biased questions. Biased questions influence a survey takers answers, making them change the responses they give you.

So what?

Well, when people don’t respond according to their own opinions but instead just tell you what you want to hear, everybody loses. The worst problem is that you get misleading data. Susan’s survey question will mean that there aren’t enough hamburgers at the party, and Walter’s will mean that there are too many. This will lead to cranky partygoers or wasted money at the grocery store. This may seem trivial when you’re thinking about a barbecue (though I’m sure Susan and Walter wouldn’t agree) but if you’re a business making decisions based on your survey data, the stakes can be huge. Cranky customers or product you can’t get rid of can spell D-O-O-M for your business. Moreover, biased questions will also lose you credibility with your survey takers. Biased questions are seen as unprofessional, and that’s bad for business. They may also give survey takers a negative impression of your brand or your company and this can lose you money and prestige.

How do you fix it?

Try to make people feel comfortable answering your questions truthfully. There are a few different ways you can do this:

  1. Remove unnecessary information. Instead of expressing your views on animal rights or American patriotism, keep it simple and direct. Something like, “How many hamburgers do you typically eat at a barbecue?” This will free people up to say what they think instead of what they think you want to hear.
  2. Balance the question. Present both sides within in the question. For example, “Do you want a hamburger or not?” This signals to people that they’re welcome to dissent with the way your question is phrased–no hard feelings.
  3. Balance the survey. Sometimes it’s hard to balance the question itself, and it has to be framed in either a positive or a negative manner. To get around this problem, just make sure that your survey has both positively and negatively framed questions in it. In Susan and Walter’s case, they could ask about both hamburger intake AND veggie burger intake.

The moral of the story–When it comes to survey writing, keep your opinions to yourself and your own biases quiet. It’s better for your data and for your image.

As for Susan and Walter’s party guests? Well, we support your barbecue item of choice, whether it be hamburgers or veggie burgers. We’re fair and balanced like that.

Questions for us? Let us know in the Comments below!

Tags: ,

One thought on “A Little Bias is a Big Problem

  1. Ma Yee Yap says:

    Yes, a survey’s purpose is to find out opinions of the target audience, not the surveyor’s!

Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help


Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide