Online Surveys = Ongoing Insights

How to Be Sure the Images in Your Survey Don’t Skew Your Results

How to Be Sure the Images in Your Survey Don’t Skew Your Results

When you’re writing a survey, you probably consider how to write your questions so they don’t bias your respondents. Because when you’ve got biased questions, you’ve got bad data. But did you know that there’s more to consider when it comes to writing a sound survey?

You also need to think through the visual presentation of your questions—including the images you choose and where you insert them in your survey. Because, as they say, a picture’s worth a thousand words. And while images can be examples you use to convey concepts to your respondents, you don’t want an ill-chosen (and ill-placed) image to influence their answers.

Need an example? Imagine you’re faced with this survey question:

In the past month, how many times have you gone shopping?

Think about how you’d go about answering this question. How do you define shopping? What you consider “shopping” may be different from the next person. And the next.

Now, take a look at this picture and answer the question again:

Shopping #1

After seeing this picture, would you give a different answer? Did you shop more than what you originally thought? Less? The same? Okay, let’s look at another picture:

Shopping #2

If you’d seen this picture of ladies shopping instead of the picture of people grocery shopping, would your answer have changed? If you think your answer would’ve been influenced by the image you were shown, you’d be in good company. A study from the University of Michigan shows that respondents indicate the highest shopping frequency when they’re presented with the picture of people shopping in a grocery store.

And when they see the picture of people shopping in a department store? Well, you guessed it—they say that they shop much less frequently. (Those who were asked the question without an accompanying image reported a shopping frequency that fell between the frequency extremes.)

So why is this the case? Respondents are using the images in the survey as examples, and the images trigger the relevant information from their short-term or long-term memory. When seeing the grocery store shopping image, respondents are likely to start counting the number of times they shopped at grocery stores last month.

In contrast, when the department store picture is shown along with the survey question, respondents will think that the question is asking about shopping for clothing, which, for most people, happens much less frequently than shopping for groceries. That’s why the image the survey-creators choose can directly affect how people answer survey questions.

So choose your images carefully lest you bias respondents. End of story, right?


The truth is, not only the content of the image can have an impact on the survey response—but its placement can make a difference as well. The same group of researchers conducted another series of experiments in which they varied the position of the images. They found that when the image was placed in the header and not near the question, it had much less impact on the responses provided by the respondents. They call this phenomenon “banner blindness.”

So what should you do when you have an image that you want to put in your survey?

  • If it’s a logo of your organization and you want to put it in the header, go ahead and do it. Because of “banner blindness,” your logo should have limited impact on data quality.
  • If it’s an image that is related to the survey topic, then you need to be more cautious. Your image may make your survey look more attractive, but at the same time your image may pose a threat to the data quality. Try to imagine how your image (like the images above) could affect how your respondents answer your questions.
  • Instead of including images in your survey, be specific in how you word a question. Think back to your survey goal. If the idea is to gather information on which products people buy at the grocery store, ask something along the lines of: In the past month, how many times did you go shopping at a grocery store? But, if your survey goal is to gather information on how much people are going to spend on their new summer wardrobe, your question might look like: In the past month, how many times did you go shopping at a department store? That way, you have primed your respondent to think about the situations in which they’re shopping, which will provide you with accurate, actionable data you can analyze.

Do you include images in your surveys? Have you noticed that different images affect the data you collect? Share your research with us in the Comments below!

Citations Used:

Couper, M. P., Tourangeau, R., & Kenyon, K. (2004). Picture this! Exploring visual effects in web surveys. Public Opinion Quarterly, 68(2), 255-266.

Couper, M. P., Conrad, F. G., & Tourangeau, R. (2007). Visual context effects in web surveys. Public Opinion Quarterly, 71(4), 623-634.
Get the Feedback eGuide

Tags: , ,

  • Elliott Brown

    Should you still use your logo in something like a brand awareness survey where you’re hoping to measure unaided brand awareness? Seems to me there are time you might want to run surveys without your logo–especially if you don’t want the survey to be associated with your brand.

    • KTsurveymonkey

      That’s a really great point, Elliott! In those cases, you may want to go with a basic theme, or simple image that isn’t associated with brand awareness.

    • Assume that anything you place in the respondent’s line of sight will be interpreted as meaningful and answers will change (consciously or not). It may not have an effect on every respondent, but design as if it does. So if logo is important to the measurement (e.g., the logos represent the brands you’re asking Rs to rate and you’re asking “Have you seen this logo?”) then use it. If it’s your company logo, you may want to think twice about it. Is your company one that “the average person” would know outside of the context of this specific survey? If not (i.e., almost any research firm), it probably isn’t a problem, and just serves to legitimize the survey. However, if you’re the American Lung Association and conducting a survey on smoking, you might not want to plaster your logo over every page of the questionnaire (i.e., respondents might feel the “doctor’s eye” and report less smoking). Corporate logos aren’t the only concern. What affect do you think the University of Michigan logo might have on responses about educational aspirations (or the perceived cost of college for that matter). But note the “banner blindness” finding Mingnan reports. A logo hiding in the corner that appears on all questions shouldn’t have the same effect as one that’s prominent, near the question itself, and appears just for that question (as if it’s part of the question).

  • Michelle V

    This is a very interesting article! Thank you. Getting to the truth can be challenging and I really appreciate your help in addressing bias. Are there times when images help reduce bias?

    • Mingnan

      It really depends on the question. If it is a attitudinal question, often times images can hurt. But if it is a factual question, images can sometimes help. The same group of researchers cited in the blog recently published another study where they use images as examples in their survey questions. They found that when the examples are atypical, they can help the respondents form a answer. For example, when asked about how many types of vegetables one had in the last week, a picture of french fries (yes, they are counted as vegetables!) will help the respondents better form a correct answer. Otherwise, without the pictures, most people won’t count french fries in their calculations.

  • Great summary! This is one of my favorite measurement error effects.

    Hope you’re enjoying your internship!

    • Mingnan

      Thank you Matt!

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in

Write Surveys Like a Pro

Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »