Online Surveys = Ongoing Insights

How to Manage Matrix Questions for Even Better Data

How to Manage Matrix Questions for Even Better Data

Survey Tips!We get it. Creating the perfect survey design that will result in easy to read, useful data can sometimes feel a bit daunting. One of the hardest parts of designing a survey is anticipating how your analysis may appear once the responses have been collected.

When sending a survey out on customer satisfaction, you want the survey experience to feel seamless for your customer, but also be easy for you to analyze later.

So how exactly do you avoid creating messy data? Well, that’s what we’re here for!

An essential part of the survey design process is choosing the right question type. Simpler question types, like Multiple Choice, almost always turn out well in analysis. But sometimes it’s necessary to branch out with our more complex question types, such as our friend, the Matrix question. This is an ideal question type when you’re asking respondents to evaluate a series of questions or statements under the same criteria. However, it takes a bit of survey design know-how to make sure you use this question type right.

With a question like the Matrix, simplicity is best. When you’re in the midst of writing a survey, and you’re up against a deadline, it’s tempting to cram a big idea into a single question. But watch out, dear survey designer—too many answer choices can have a huge affect on the quality of your data.

Screen Shot 2015-03-12 at 9.54.48 AM

In the example above, you’ll see that it’s easy to get lost in this sea of choices. If this is a survey sent to your customers, you certainly wouldn’t want them getting frustrated when considering how satisfied they are with your product. A poorly designed Matrix question can lead the respondent to rush through your question just to get it over with, ultimately leading to weaker data.

When it comes time to analyze the data from your Matrix question, your chart will end up looking like the one below. Yikes.

q1 analyze ss redo

With so many data points, where do you start? When comparing how customers feel about their level of support, the implications can leave you unable to accurately depict their preferences.

If you’re ever in a pickle and need to include more answer choices, try splitting one big question into multiple smaller question. Using three different question categories of “customer service rep”, “timeliness” and “overall experience”, allows you to make more pointed statements about your data. As a comparison, you’ll find that with less answer choices your analysis experience will be far simpler and discerning what’s important in your data will be more quickly apparent.

Screen Shot 2015-03-12 at 9.50.37 AM

This is also true when adding columns to your Matrix question. The difference in scale between “Agree” and “Strongly Agree” quickly weakens as choices are added. We generally recommend sticking with a 3 to 5 point scale. It’s also important to think about what sort of device your respondent will be taking the survey on. If you expect most of your customers to be using their mobile device, condensing these Matrix questions or using a Multiple Choice question type will be your best bet.

Now that you’ve taken our crash course on Matrices, you can start using this question type with confidence. This can help you create experience for your customers so that evaluating your product or service becomes effortless. To top it off, you can rest assured that you’ll have a beautiful analysis to look forward to!

Questions for Harrison? Let him know in the Comments below.

Get the Feedback eGuide

Tags: , , , ,

  • FujiS2500HD

    This is very much on point. If I see a giant matrix like the one in the example, I’m Christmas-Tree-ing it and moving on.

    Also, protip: the “please select option 6 for data verification purposes” questions are super easy to spot and don’t put off survey speeders one bit.

    Add timing devices into surveys and warn that speeding through will disqualify them. If you HAVE to include a giant ass matrix, add code so the next row only appears once you’ve selected an option in the previous one. This doesn’t overwhelm the user from the get-go.

    Progress bars (and ACCURATE ones) help the user know how much they’ve got left.

    And, of course, remember kids: reward survey-takers well and keep the questions simple. Those 2 tips will give you your best results.

  • David Murray

    I noticed on the likert scale used in the post that Disagree was placed on the left hand side before Agree. What’s the reason for that? Thanks

    • KTsurveymonkey

      Hi David,

      Good question. Part of the reason is that you can use weighted averages with a Matrix question. So in Harrison’s example, disagree has a weight of 1, and agree holds a weight of 5. Also, It’s really a design preference. There’s not really a right or wrong way of labeling in terms of agree to disagree. It’s perfectly acceptable, and sometimes preferred, to label agree to disagree!

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in
Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »