Online Surveys = Ongoing Insights

New Advanced Logic Feature: Question Randomization

New Advanced Logic Feature: Question Randomization

Over the last week, I’ve been sharing some updates on our new advanced logic features.  Now that you’ve seen what piping can do, and how random assignment works, it’s time for us to tackle Question Randomization.

Why randomize your questions?

One issue survey creators need to consider when designing a survey is question order effect.  Earlier this week, Dr. Phil Garland explained why question order matters in his blog post on the topic.  Essentially, by displaying questions or content in a particular order, it can affect how people respond to your survey.  For example, if you have the following questions in the following order:

Q1 What is the best thing about living in your community?
Q2 How satisfied are you with how your local politician is handling her job?

Just answering Question 1 can affect how people answer Question 2. To help mitigate this, you can choose to randomize your questions.

To see question randomization in action, check out the sample survey embedded below.  Refreshing this page will show how the order of the questions changes.

Create your free online surveys with SurveyMonkey, the world’s leading questionnaire tool.

How to implement question randomization.

Create a page of questions, then click “Edit Page Options” and choose “Add Question Randomization.”

You can choose to randomize:

  • All Questions: shows all questions on the page in random order
  • Selected Questions: lets you choose which questions on the page you want to randomize
  • None: lets you remove randomization
  • Flip: lets you flip the first and last question on the page

And there you have it.  Reduce question order effect in your survey with a few clicks.  If you have questions on how to use this new feature, please ask me in the comments below.

Tags: , , ,

  • Phil Kortum

    Can you randomize pages instead of questions? We have a survey structure that has participants rate pictures along multiple dimensions and while we want to keep those questions in the same order on each page, we want to randomize the pages (picture +questions group). Is that possible? Thanks in advance for your answer.

  • Azhar Khan

    can I publish 10 questions randomly out of 100 questions pool and every time it publish 10 random questions, it also publish 10 random answers like for example
    Q 1 : how many days in a week
    A: 7
    B: 6
    Next time when i publish this question it should publish with different order of answers
    Q1: How many days in a week
    A: 6
    B: 7
    Please let me know if i can do that.. if yes then i will be interested in buying this service

    • Kayte K

      Hi Azhar! Thanks for your question. Unfortunately, we don’t offer randomization options for publishing questions out of a pool of questions. Hope that helps and let us know if you any more questions!

  • Sylva

    Are the questions randomized each time a survey is taken? Or do they get randomized one time when we submit our questions?

    • Kayte K

      Hi Sylva- great question! They’re randomized each time a survey is taken.

  • Andrea

    is it possible to store the (random) order with which every user has taken the survey? This would be important to see how answers change when the order of the questions is changed.

    • KTsurveymonkey

      Hi Andrea!

      Great question. The order in which your respondents see the questions will be visible from the individual responses tab within Analyze Results. You can click on the [Edit Responses] option, and you can click through the survey in order to see the order the respondent saw. Unfortunately there isn’t any other direct way to store the randomization at this time.

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in

Write Surveys Like a Pro

Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »