Ask more, know more, do more

Question Order Matters

Question Order Matters

The order questions appear in your survey can directly impact the responses you gather. One of the more well known examples (Iyengar & Kinder, 1987) of question order effects is in the domain of politics. When pollsters ask people, “What is the most important problem facing the nation?,” the answer they give becomes the object of focus for their answer to a subsequent question—“Do you approve or disapprove of the way [Barack Obama] is handling his job as president?” By and large people answer the approval question while judging the president primarily on his performance on the issue they consider to be most important.

This phenomenon is called priming. Respondents are primed to think about one issue while answering the subsequent question.

Priming has an impact even if the survey topic is not political, or even controversial. Let’s take a seemingly harmless survey. Let’s say I’m trying to gauge sentiment for a company softball team. I may be tempted to ask “What is your favorite sport?” as part of my survey. Seems like a fair start. But if my next question is, “How interested are you in playing on a company softball team?” and softball isn’t high on your list of favorite sports, then you may be tempted to rate your interest lower in joining the company team than if I had asked you first about the team, then about your favorite sport.

Another reason question order matters is that respondents may have a desire to appear consistent in their responses. For example, if you ask students to answer a very difficult math problem first, then ask how much they enjoy math, they may be tempted to rate their interest lower if they struggled to solve the math problem.

Even response options from an earlier question can impact subsequent answers. If I ask “Which of these four fruits is your favorite?” and then ask how much fruit you eat in a week, you may focus your attention on just those four items and report a lower number than if you were to think of a longer list of fruit.

How can you address question order effects in your own surveys? One option is to randomize your questions so that respondents are not all answering questions in the same order. My colleague, Tim, will be posting this week on one of our NEW advanced logic features, Question Randomization, that allows you to do just that—randomize questions to reduce question order effect. Have additional questions on why question order matters? Please feel free to ask me in the comments below.

Do a Better Job with SurveyMonkey Enterprise

Give yourself—and the rest of your team—the answers you need to make smarter decisions. Access our most powerful tools with SurveyMonkey Enterprise.

Learn More →

Tags: , , ,

18 thoughts on “Question Order Matters

  1. Jeroen Egelmeer says:

    At the moment I’m busy with my graduation project about a survey from a large company. Now I think that one questions in the should be asked as first to avoid a bias. (it’s now on the fourth place) The only problem I have is to convince the company to put that specific question to the first place. So my question is, do you have some scientific information for me to convince the company to change te place of the question?

  2. Felicity says:

    I would like to design a survey where the answers to some of the questions will be in other questions later on. Therefore, it is important that people cannot go back and change their answers. Is this possible?

    1. Anne R says:

      Hi Felicity–it definitely is. There are two features you can use together to do this. The first is Piping (that will let you put answers from previous questions into subsequent questions). The second is to set up your survey collector so that it doesn’t allow respondents to change their answers. For more info on Piping:, and for more info on how to prevent respondents from changing responses: Let us know if you need anything else!

  3. dimondn says:

    I am new to surveys but wondering if someone might opt out of a survey if unable to go back and adjust answers to questions.
    Is that a factor?

  4. Matt says:

    I am in the process of creating a survey using several already existing measurement scales. However, I have to counterbalance the order in which each scale is given without changing the order of the questions within each scale itself. Is there a way for me to do this?

    1. Phil G says:

      Hi Matthew,

      It sounds like you want to minimize order bias by offering flipped or rotated response options, at random, to your respondents. Flipping involves presenting answer choices as either 1, 2, 3, 4 or 4, 3, 2, 1. Rotation requires showing, for example an order of 1, 2, 3, 4, 5 for the first respondent and 2, 3, 4, 5, 1 for the second respondent and 3, 4, 5, 1, 2, for the third respondent etc. Both methods guard against satisficing.

      Both of these options are available in SurveyMonkey.

  5. Susan says:

    I would like to create a brief survey regarding students visits to the tutoring lab that I oversee. I have approximately six questions, but any feedback on setup would be appreciated. I am looking at such things as helpfulness of the tutor, convenience of lab hours, how can the tutoring be improved, and so forth.
    Thank you!

    1. Bennett P says:

      Hi Susan –

      Try our new Question Bank. When you go to create the new survey, take a look at the Question Bank options on the left.
      Or, you can use one of our expert survey templates. I’m not sure if we have one that fits the tutoring lab exactly, but the University Student Satisfaction might work or at least give you a starting point.

  6. Sandy K says:

    How do I get construct my survey so that Survey Monkey gives me a total point value and the end of the survey. For example, we are asking questions with a 1 to 5 point value. There may be 15 questions and I want to know the points scored for each question and the total points for all questions. For example If someone strongly agrees that is 5 points. Also is it possible to give a weighting percentage factor to the questions and have Survey Monkey provide the weighted total?

    1. MychalH says:

      Hello Sandy –

      Take a look at this FAQ on how you can transform the data numerically when exporting the data into Excel:

      If you have any additional questions, please send them to so we can best assist you.

  7. Minoo says:

    Do you have any recommendations that how many randomised versions of a questionnaire should be produced in order to reduce the bias? (e.g. for 10 questions with 100 sample size).

    1. Hanna J says:

      What size sample you need depends on what population you’re trying to estimate, how accurate you need the estimate to be, and how many variables you’re estimating.
      For example, for our population of roughly 200,000 pro subscribers we would need at least 750 responses in our sample to estimate ONE question.
      Want to add more questions? You’ll need more sample. (For a basic estimator tool to figure out your own sample needs, check out this page:

  8. Brashani says:

    I am currently working on a research project where we need to counter-balance two portions of the online survey. Does SurveyMonkey allow you to do this? Thank you!

    1. Hanna J says:

      Hi Brashani – We offer a Random Assignment feature which helps control for ordering effects. This feature, available to our Gold and Platinum users, lets you create different versions of your survey. We then randomly assign people to one of them. Read more here:

      Let us know if you have any more questions. Thanks!

  9. Anonymous says:

    I have really learned new things from your blog post. Yet another thing to I have noticed is that in most cases, FSBO sellers will reject a person. Remember, they will prefer to not use your services. But if a person maintain a gentle, professional connection, offering assistance and staying in contact for about four to five weeks, you will usually have the capacity to win a conversation. From there, a listing follows. Cheers

    1. Kayte K says:

      We’re glad you found this helpful and thank you for the feedback!

Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help


Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide