Online Surveys = Ongoing Insights

Now Analyzing A/B Testing Survey Results is Easier than Ever

 → 
 → 
Now Analyzing A/B Testing Survey Results is Easier than Ever

Screen Shot 2014-05-01 at 4.51.16 PMExciting news, survey fans! We’ve added some new features that make it easier than ever to analyze the results of an A/B test survey. Now, whenever you include A/B testing in your survey (which shows respondents text, image, or question variants) you can quickly compare the differences in your respondents’ answers based on which version they saw.

What do we mean? Here’s a simple example. Let’s say you’re Coca-Cola and you want to pre-test two logos to see how engaging and professional people think they are.

First, set up an Image A/B Test in your survey using your two images. (Note: Make sure to add text descriptions and a test nickname to make it easy to analyze the results.)

Edit Question A:B

 

Next, follow that randomized image with two questions about the image. The first asks “How engaging is the logo?” and the second asks “How professional is the logo?”

Each respondent will only see one of the logos and will answer the questions based on the logo they saw.

After receiving responses, go to the Analyze Results tab in SurveyMonkey to see your results. You’ll get a look at the overall totals for the questions about the logo.  Let’s say for the first question, you get results like this:

Image3

These aggregate results are interesting, but what you really need to know is how engaging the respondents found each logo. Take a look at the Saved Views section to the left of your results. For each A/B test that you create in your survey, SurveyMonkey automatically creates a saved view.

Image4

In this example, the automatically-created Saved View is named with the nickname you gave your A/B Test when you designed your survey. Here, it’s Logo Test.

When you click the Saved View named Logo Test, you see that SurveyMonkey has automatically created a Compare rule to compare your results by the variables in your image A/B test.

Image5

Now when you look at the results for your two questions, you get to see them side-by-side—so it’s easier to figure out which logo your respondents thought was more engaging.

Image6

Clearly, the picture of a bottle was more engaging than just the words, “Enjoy Coca-Cola”. (And you also get the exact numbers in the data table.)

Image7

SurveyMonkey also provides other tools to help analyze A/B testing results. If you’re only interested in the response to one of the logos, you can filter results by variable. To do that, go to the Filter rule builder and select Filter by A/B Test.

Image8

(Note: You can also create your ownCompare rule by selecting Compare by A/B Test under Compare in the rule builder.)

Next, choose which A/B Test you want to use for your filter.

Image9

After selecting Logo Test, choose which of the image variables you want to use to filter your results.  In this case, check Bottle Logo to see only the responses from people who saw the bottle logo in their surveys.

Image10

When you apply this filter, you get filtered results and can see how all the respondents who saw the bottle logo answered any of the questions:

Image11

As you can see, respondents thought the bottle logo was very engaging.

With these new analysis tools, A/B tests are easy to create and even easier to interpret after the responses come in. And don’t forget, you can always use SurveyMonkey Audience to get a targeted random sample of respondents to answer your A/B testing surveys.

Have you run an A/B test lately? Tell us about it in the Comments below!

Tags: , , , , , , , , ,

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in
Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »