We already know adding images to your survey can play a major role in how people answer your questions. But can images in your survey invitations affect who decides to respond?
You bet. The U.K. elections in May gave us the perfect opportunity to get new data that show how images in survey invitations can impact whether or not people respond to your survey, be it via your website or survey email invitation.
The experiment: Can images affect performance?
We conducted a survey experiment in March 2015 that asked U.K. residents which political party they would vote for in the May election for Parliament.
We randomly displayed one of three survey invitation designs to potential respondents. In the first image, we presented a picture of 10 Downing Street, the U.K. Prime Minister’s office and residence. We asked recipients, “Who do you want for the top job?”
The second image showed a U.K. voter map color-coded by party representation that asked respondents, “Which way will you swing in May?”
The third image presents five rosettes, which are ribbons often used for military decoration. We used the same call to action for this image as we used in the map image.
The results: Your images affect who clicks
After sending out these different images and calls to participate in the survey, we looked at variations in click rate (people who clicked the “Take the Survey” button); completion rate (people who made it all the way through the survey and submitted their answers); and the actual survey responses.
Since this experiment was done for a web survey, we were most interested in whether adding images affected click rate and survey completion rate.
And we found that including images on survey invitations appeared to impact click rate the most. The voting map, for example, performed the best of the three images with a 9% click rate. It was closely followed by the rosettes image with 8%, while the 10 Downing Street image had just a 6% click rate:
There were far fewer differences in completion rates. But the completion rates for all three were quite high—around 90%:
In politics, demographics matter
Since the survey was mainly about the U.K. election, we asked several political questions and looked at whether their responses differed between each of the invitations. The 10 Downing Street image drew significantly more respondents who identified with the Conservative Party and were more likely to vote for the Conservative Party in the upcoming election.
A possible explanation for this difference is that the Prime Minister David Cameron is the Conservative Party leader and that conservative-leaning respondents may have been motivated to participate in the survey when they saw the image of 10 Downing Street image, where Prime Minister Cameron lives and works. This indicates that survey respondents could differ depending on what image you use in your survey invitation.
The survey also included a few questions on how respondents consume media, and we turned up some interesting differences. Respondents recruited through the 10 Downing Street image were less likely to use social media and were more likely to watch TV.
In order to find out why, we compared the demographics of respondents who saw each of the images and noticed that those who got the 10 Downing Street image tended to be older than the other two survey invitation webpages. The age differences could be a major contributor to the media usage differences.
What’s the takeaway from this experiment?
Using images in survey invitations can influence the following:
- The likelihood that the invited participants will click on a survey invitation and participate in your survey
- The type of people who respond to your survey
- The survey responses—especially when the questions and images are closely related to respondent characteristics
When writing your surveys, think carefully about which images to use in your survey and your survey invitations. You may want to think about whether you want to use images in the first place. Wherever you include them, images are powerful elements in surveys that significantly affect your results.
If you are designing a survey for a specific group of people and you can identify an image that appeals to that group, using it in the survey invitation may help improve the response rate. Otherwise, if you are doing a survey targeting general population, you need to be more cautious on choice of images.