Online Surveys = Ongoing Insights

Don’t Let Your Own Opinions Sneak Into Your Survey: 4 Ways to Avoid Researcher Bias

Don’t Let Your Own Opinions Sneak Into Your Survey: 4 Ways to Avoid Researcher Bias

Avoid BiasBias is the mortal enemy of all surveys, and as a survey creator it’s important to guard against it to make sure you get reliable results. Over the years we’ve developed many tips to promote honest answers from your respondents, and best practices for designing surveys that use unbiased wording, structure, and styling. But if you’re not careful, there are a few ways you can still introduce bias without even knowing it.

One of the leading causes of misleading survey data is researcher bias that comes directly from the survey writer. This bias is sneaky. It’s caused by survey creators who innocently influence the results to reach an outcome they hope or expect to reach. It’s sneaky because survey creators are typically unaware it’s happening.

Can you avoid this kind of bias?

Don’t worry! We have you covered with a researcher bias prevention list. Below are four things to avoid when conducting your survey:

1. Asking the wrong questions

It’s impossible to get the right answers if you ask the wrong questions. Unfortunately, survey results are easily compromised by questions that fall short of capturing the entire scope of a survey’s issue. Say, for example, your survey was created to understand your employees’ favorite type of pizza. You ask, “Do you like pepperoni, meat lovers, or vegetarian pizza the best?” Though there are many other types of pizza, they did not come to your mind and were left out of the question. Now instead of measuring the most popular pizza, the study measures the preference between these three types.

Tip: Exploratory research is the best way to make sure your questions are exhaustive and on point. By first surveying a small group with open-ended questions on your subject, you’ll gain a better perspective of the scope of your survey topic, and be less likely to overlook options that may matter to respondents. You may also want to review similar surveys to learn what categories and topics were popular with respondents in the past.

2. Surveying the wrong people

Choosing your respondent group may seem like a no-brainer, but it often leads to something called selection bias. When conducting a survey, it’s imperative to target a population that fits your survey goals. If you incorrectly exclude or include participants, you may get skewed data results.

Usually this bias happens when you lack of a clearly defined target population. For example, say you want to limit your survey to people with low economic standings. This population could be defined in many ways: people with low income, people who lack disposable income, or people who have a low net worth after taking into account their property, income, and debt. Each of these three descriptions can successfully be used to address the broad population you hope to reach. But, each definition could provide different results for your study.

Tip: To avoid surveying the wrong people, make sure you clearly define the respondent requirements you need to meet your survey objectives before beginning your project. This step will give your survey results a proper scope. Also remember to be specific in your reports and findings when referring to your population. Using broad terms like poor, rich, large, or small can lead to misinterpretation.

3. Using an exclusive collection method

Some surveying methods can make it difficult, or even impossible, for certain people to take part in your study. For example, if you survey commuters you meet walking around on the street, you might not get a representative sample of people who drive or ride bicycles. By excluding potential respondents in a non-random way, you can instill bias into your survey if the people who aren’t part of the panel have views that differ than those who are.

Tip: The best way to limit this type of researcher bias is to give all potential respondents an even chance to participate in your survey. In our commuter example, you might be better off sending an online survey to everyone who lives in your town, or asking some local businesses to send your survey out to all their employees.

Need the Right People to Take Your Survey?

We’ve got millions of survey respondents ready to tell you what they think. Choose from 30+ profiling criteria.

Get Started

4. Misinterpreting your data results

This form of bias is introduced when raw data is transformed into misinterpreted findings. Usually it’s a case of inappropriate or inaccurate statistical techniques, which lead to the incorrect interpretation of the survey results. For example, bias can come into play when a survey creator gets excited about a finding that meets their hypothesis but overlooks the fact that the survey result is only based on a handful of respondents.

Tip: Most often, this form of bias is caused by gathering information and then later developing your data analysis strategy. To avoid this type of bias, create a data analysis plan before you write your survey, then write questions that you know will work well with the analysis you have in mind. For example, try creating multiple choice questions with rating scales that will work well when you want to quantify your results. It also helps to take note of the different analytical tools available to you from your survey software before you create your survey.

Avoiding these four mistakes may seem difficult at first, but by remaining true to your survey’s purpose and having a firm understanding of the topics of your research you’ll be well on your way to eliminating researcher bias from your survey.

Put simply: You need to do some planning before you start your survey. Take a second to think about each of the four points above and whether your survey plan addresses them. Once you’re sure that your research methodology is sound, you can rest easy that your final results won’t lead you astray.

Questions for Rick? Let us know in the Comments below!

Get the Survey Writing eGuide


Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help


Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide