Online Surveys = Ongoing Insights

Don’t Let Your Own Opinions Sneak Into Your Survey: 4 Ways to Avoid Researcher Bias

Don’t Let Your Own Opinions Sneak Into Your Survey: 4 Ways to Avoid Researcher Bias

Avoid BiasBias is the mortal enemy of all surveys, and as a survey creator it’s important to guard against it to make sure you get reliable results. Over the years we’ve developed many tips to promote honest answers from your respondents, and best practices for designing surveys that use unbiased wording, structure, and styling. But if you’re not careful, there are a few ways you can still introduce bias without even knowing it.

One of the leading causes of misleading survey data is researcher bias that comes directly from the survey writer. This bias is sneaky. It’s caused by survey creators who innocently influence the results to reach an outcome they hope or expect to reach. It’s sneaky because survey creators are typically unaware it’s happening.

Can you avoid this kind of bias?

Don’t worry! We have you covered with a researcher bias prevention list. Below are four things to avoid when conducting your survey:

1. Asking the wrong questions

It’s impossible to get the right answers if you ask the wrong questions. Unfortunately, survey results are easily compromised by questions that fall short of capturing the entire scope of a survey’s issue. Say, for example, your survey was created to understand your employees’ favorite type of pizza. You ask, “Do you like pepperoni, meat lovers, or vegetarian pizza the best?” Though there are many other types of pizza, they did not come to your mind and were left out of the question. Now instead of measuring the most popular pizza, the study measures the preference between these three types.

Tip: Exploratory research is the best way to make sure your questions are exhaustive and on point. By first surveying a small group with open-ended questions on your subject, you’ll gain a better perspective of the scope of your survey topic, and be less likely to overlook options that may matter to respondents. You may also want to review similar surveys to learn what categories and topics were popular with respondents in the past.

2. Surveying the wrong people

Choosing your respondent group may seem like a no-brainer, but it often leads to something called selection bias. When conducting a survey, it’s imperative to target a population that fits your survey goals. If you incorrectly exclude or include participants, you may get skewed data results.

Usually this bias happens when you lack of a clearly defined target population. For example, say you want to limit your survey to people with low economic standings. This population could be defined in many ways: people with low income, people who lack disposable income, or people who have a low net worth after taking into account their property, income, and debt. Each of these three descriptions can successfully be used to address the broad population you hope to reach. But, each definition could provide different results for your study.

Tip: To avoid surveying the wrong people, make sure you clearly define the respondent requirements you need to meet your survey objectives before beginning your project. This step will give your survey results a proper scope. Also remember to be specific in your reports and findings when referring to your population. Using broad terms like poor, rich, large, or small can lead to misinterpretation.

3. Using an exclusive collection method

Some surveying methods can make it difficult, or even impossible, for certain people to take part in your study. For example, if you survey commuters you meet walking around on the street, you might not get a representative sample of people who drive or ride bicycles. By excluding potential respondents in a non-random way, you can instill bias into your survey if the people who aren’t part of the panel have views that differ than those who are.

Tip: The best way to limit this type of researcher bias is to give all potential respondents an even chance to participate in your survey. In our commuter example, you might be better off sending an online survey to everyone who lives in your town, or asking some local businesses to send your survey out to all their employees.

Need the Right People to Take Your Survey?

We’ve got millions of survey respondents ready to tell you what they think. Choose from 30+ profiling criteria.

Get Started

4. Misinterpreting your data results

This form of bias is introduced when raw data is transformed into misinterpreted findings. Usually it’s a case of inappropriate or inaccurate statistical techniques, which lead to the incorrect interpretation of the survey results. For example, bias can come into play when a survey creator gets excited about a finding that meets their hypothesis but overlooks the fact that the survey result is only based on a handful of respondents.

Tip: Most often, this form of bias is caused by gathering information and then later developing your data analysis strategy. To avoid this type of bias, create a data analysis plan before you write your survey, then write questions that you know will work well with the analysis you have in mind. For example, try creating multiple choice questions with rating scales that will work well when you want to quantify your results. It also helps to take note of the different analytical tools available to you from your survey software before you create your survey.

Avoiding these four mistakes may seem difficult at first, but by remaining true to your survey’s purpose and having a firm understanding of the topics of your research you’ll be well on your way to eliminating researcher bias from your survey.

Put simply: You need to do some planning before you start your survey. Take a second to think about each of the four points above and whether your survey plan addresses them. Once you’re sure that your research methodology is sound, you can rest easy that your final results won’t lead you astray.

Questions for Rick? Let us know in the Comments below!

Get the Survey Writing eGuide


  • Shane Miller

    I know you guys are customer focused… but it would be nice somewhere on your website if there were a “contact us” form. YES… that would cost you some money to have someone answer those emails, but that’s what a “Customer focused” company does!

    • KTsurveymonkey

      Hi there, Shane!

      Sorry for any confusion here. We actually have a large customer support team ready and willing to answer your customer questions 24/7! The contact us form is available at the bottom of our Help page, or here: They’ll be more than happy to help out with any questions you have.

      • Maria Masters Harbert

        Hey KT,

        Maybe there’s a survey for Survey Monkey here regarding customer service. Most of us using SM have businesses and give a lot of thought to how easy we want to make it for customers to contact us. We can place a “contact us” link in our website footer or we can place it at the top of our page. We can put our phone number on our contact us page or just an email form. And then some of us put our toll-free number at the top of our website and in our Google Adwords mobile ad. Maybe I’m off here, but this is something I’ve given thought to based on how much do I really want people calling us and/or emailing us. Placing your contact email at the bottom of your Help Page kind of looks like you’re hiding your contact information, and doesn’t convey a customer-service minded company to many users. Why not put a link to that person who’s “ready and willing to answer your customer questions 24/7” in your footer so it’s quick and easy to find? Just my thoughts and opinion. Perhaps a survey would reveal otherwise.

        • KTsurveymonkey

          Hi there, Maria!

          Definitely fair and valuable feedback. Your thoughts around this make a lot of sense. We put a lot of hard work into our incredibly vast help center. It can sometimes be faster to find the information you need there, and contacting support can sometimes take more time. Of course, our support team is definitely here for you 24/7 and will get back to you often times within an hour! We’ll be sure to pass along your feedback regarding the placement of our contact form. Thanks for taking the time to give us your two cents!

  • I have used the email contact dozens of times. I am always surprised as to how quickly my questions get a response. I have even had dialogues that need two or three email exchanges and they happen in less than 15 minutes. With other companies I have been on hold longer than that just waiting for someone to answer.

  • Suzanne Jackson

    On point number 1 on above, you forgot to include an option for “I don’t like/care for pizza”. That is the BIG thing overlooked is to find out if a customer doesn’t even like or use the product.

  • Søren Madsen

    I was wondering if SurveyMonkey has access to respondents within the fundraising segment? I do an international survey at at the moment and need to find 100 professionel fundraisers

    • KTsurveymonkey

      Hey Soren!

      You can reach out to our Audience sales team and they’ll be happy to check that out. They can be reached at 🙂

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in

Write Surveys Like a Pro

Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »