Online Surveys = Ongoing Insights

How to Be Sure the Images in Your Survey Don’t Skew Your Results

 → 
 → 
How to Be Sure the Images in Your Survey Don’t Skew Your Results

When you’re writing a survey, you probably consider how to write your questions so they don’t bias your respondents. Because when you’ve got biased questions, you’ve got bad data. But did you know that there’s more to consider when it comes to writing a sound survey?

You also need to think through the visual presentation of your questions—including the images you choose and where you insert them in your survey. Because, as they say, a picture’s worth a thousand words. And while images can be examples you use to convey concepts to your respondents, you don’t want an ill-chosen (and ill-placed) image to influence their answers.

Need an example? Imagine you’re faced with this survey question:

In the past month, how many times have you gone shopping?

Think about how you’d go about answering this question. How do you define shopping? What you consider “shopping” may be different from the next person. And the next.

Now, take a look at this picture and answer the question again:

Shopping #1

After seeing this picture, would you give a different answer? Did you shop more than what you originally thought? Less? The same? Okay, let’s look at another picture:

Shopping #2

If you’d seen this picture of ladies shopping instead of the picture of people grocery shopping, would your answer have changed? If you think your answer would’ve been influenced by the image you were shown, you’d be in good company. A study from the University of Michigan shows that respondents indicate the highest shopping frequency when they’re presented with the picture of people shopping in a grocery store.

And when they see the picture of people shopping in a department store? Well, you guessed it—they say that they shop much less frequently. (Those who were asked the question without an accompanying image reported a shopping frequency that fell between the frequency extremes.)

So why is this the case? Respondents are using the images in the survey as examples, and the images trigger the relevant information from their short-term or long-term memory. When seeing the grocery store shopping image, respondents are likely to start counting the number of times they shopped at grocery stores last month.

In contrast, when the department store picture is shown along with the survey question, respondents will think that the question is asking about shopping for clothing, which, for most people, happens much less frequently than shopping for groceries. That’s why the image the survey-creators choose can directly affect how people answer survey questions.

So choose your images carefully lest you bias respondents. End of story, right?

Nope!

The truth is, not only the content of the image can have an impact on the survey response—but its placement can make a difference as well. The same group of researchers conducted another series of experiments in which they varied the position of the images. They found that when the image was placed in the header and not near the question, it had much less impact on the responses provided by the respondents. They call this phenomenon “banner blindness.”

So what should you do when you have an image that you want to put in your survey?

  • If it’s a logo of your organization and you want to put it in the header, go ahead and do it. Because of “banner blindness,” your logo should have limited impact on data quality.
  • If it’s an image that is related to the survey topic, then you need to be more cautious. Your image may make your survey look more attractive, but at the same time your image may pose a threat to the data quality. Try to imagine how your image (like the images above) could affect how your respondents answer your questions.
  • Instead of including images in your survey, be specific in how you word a question. Think back to your survey goal. If the idea is to gather information on which products people buy at the grocery store, ask something along the lines of: In the past month, how many times did you go shopping at a grocery store? But, if your survey goal is to gather information on how much people are going to spend on their new summer wardrobe, your question might look like: In the past month, how many times did you go shopping at a department store? That way, you have primed your respondent to think about the situations in which they’re shopping, which will provide you with accurate, actionable data you can analyze.

Do you include images in your surveys? Have you noticed that different images affect the data you collect? Share your research with us in the Comments below!

Citations Used:

Couper, M. P., Tourangeau, R., & Kenyon, K. (2004). Picture this! Exploring visual effects in web surveys. Public Opinion Quarterly, 68(2), 255-266.

Couper, M. P., Conrad, F. G., & Tourangeau, R. (2007). Visual context effects in web surveys. Public Opinion Quarterly, 71(4), 623-634.
 
Get the Feedback eGuide

Tags: , ,

Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help

 

Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide