Online Surveys = Ongoing Insights

How to Make Sense of Open-ended Responses

 → 
 → 
How to Make Sense of Open-ended Responses

Open-ended responsesGreetings, dear customers! The brand new Survey Research team here at SurveyMonkey is making its official debut on the blog today and we couldn’t be more excited to share best practices and tips with all of you. Let’s jump right in, shall we?

Using SurveyMonkey Audience–a powerful tool for targeting a specific demographic–we recently launched a political survey in seven swing states asking people their opinions on the Affordable Care Act aka “Obamacare”. Each of these states was identified by the nonpartisan online newsletter, Cook Political Report, as having competitive statewide races in the 2014 election year.

We wanted to dig deeper into one of the open-ended questions we asked on the survey, and let you in on how you can make sense of those responses and ensure the quality of your data. After all, smart data leads to even smarter decisions.

Here’s our open-ended question: “In a few words how would you describe your feelings about the health care law known as Obamacare?” And a peek at some of the responses we received:

Open-ended Response Question Type

One of the great things about open-ended questions? People aren’t limited to a predetermined set of possible answer choices so you end up collecting a rich pool of genuine opinions from folks on your survey topic. However, they also present an analytical challenge–just how do you make sense of all these unique answers?

For starters, thanks to our partnership with NVivo you can easily import your SurveyMonkey data into the NVivo platform to analyze the text. Alternatively, you can do a basic analysis right inside our Analyze tool and categorize the responses to provide not only a detailed picture of what people’s opinions are in their own words, but also to know how many people feel that way. To use the Categorize feature, just tick the box next to each response in order to place it into a category. So! Ready to become an open-ended expert?

Do a Better Job with SurveyMonkey Enterprise

Give yourself—and the rest of your team—the answers you need to make smarter decisions. Access our most powerful tools with SurveyMonkey Enterprise.

Learn More →

Five tips to get you started.

  1. Read through a couple of responses to get a sense of what folks are saying. As always, know your data. By glancing at the images of the responses above, you’ve already done the first step.
  2. Map out a few general categories to put each of the responses in. As you read the responses above, you can already see that folks are pretty much divided into three camps–those that like the law, those that hate it, and those who don’t have an opinion either way–so we created three categories: positive opinions about the law, negative opinions, and neutral opinions.
  3. Create sub-categories underneath your general ones to provide even richer detail. As you move along and put responses in their respective buckets (positive, negative, or neutral in this example), you’ll see some reoccurring themes popping up within each general category. Group and tag those themes together to create new sub-categories. For example, we saw lots of respondents saying that the law was “socialistic”, so we created a sub-category for that underneath “negative feelings”. This extra information is totally optional, so if you don’t need this level of detail, feel free to skip on down to step 4.
  4. Double check and re-categorize. Go back and re-read responses to make sure they properly fit in the categories that you’ve assigned them to. If you’ve added sub-categories along the way, you’ll be able to tag previous responses to the new sub-categories on this second look-through. Also, don’t be afraid to assign multiple categories to one response since often open-ended comments cover more than one category. But note that since each response can be assigned to multiple categories, your percentages may not add up to 100.
  5. Put a number on it! Congrats, you’re done! Click over to the My Categories tab in your question summary to see the percentages. After finishing our open-end response analysis in our political survey, we found that among those we surveyed–35% had positive things to say about the law, 55 percent negative and 10 percent had neutral comments, similar to national opinion on the law.

Since we added extra sub-categories, we were able to tell a more detailed story than if we had only looked at positive, negative and neutral as categories. For example, 10% of folks mentioned that the law was a good idea, but that due to the technical problems with the roll-out of the Obamacare website, the law needed work.

That’s it for today, everyone. We hope these tips help out the next time you want to make sense of your open-ended responses and don’t be shy, let us know your thoughts!

We’re looking for more of your questions on all things Survey Science. Need advice on how to keep your next survey project methodologically sound? Please let Sarah and the Survey Research team know in the Comments section below.
 
Get the Feedback eGuide

Tags: , , , , ,

3 thoughts on “How to Make Sense of Open-ended Responses

  1. disqus_VnnhKzNqV0 says:

    Good stuff, had me reading, researching and thinking

  2. Shirley says:

    I guess my question relies more on what resources available once using NVivo? I have 5 open ended questions and 24 respondents. Which I have already use survey monkey to gain the response.
    My real questions are:
    Cost?
    Help using the system?
    Thanks
    Shirley

    1. kaytek says:

      Hi Shirley,

      You can always write us at support@surveymonkey.com with your technical questions. To contact NVivo directly, here is their help page: http://www.qsrinternational.com/contact.aspx?webSyncID=4a06ce11-9c8f-451d-a5cd-919733e9a78b&sessionGUID=d62340bd-3d9a-4ef0-b623-4262963a2093

      Thanks,
      Kayte

Leave a Reply

Your email address will not be published. Required fields are marked *

Inspired? Create your own survey.

Make your own survey

Sign In

Sign up to get started

PRO Sign Up 
Sign Up FREE

Latest Features

Curious about what we’ve been working on?

View New Features

Get expert survey help

Get expert survey help

 

Best practices for planning expert surveys

Planning makes writing your survey easy.

Download eGuide

How many responses do you need?

Use our sample size calculator to find out!

Calculate now

Analyze survey data like a pro

Learn to slice and dice data using the Analyze tool.

Download eGuide