Products

SurveyMonkey is built to handle every use case and need. Explore our product to learn how SurveyMonkey can work for you.

Get data-driven insights from a global leader in online surveys.

Integrate with 100+ apps and plug-ins to get more done.

Build and customize online forms to collect info and payments.

Create better surveys and spot insights quickly with built-in AI.

Purpose-built solutions for all of your market research needs.

Templates

Measure customer satisfaction and loyalty for your business.

Learn what makes customers happy and turn them into advocates.

Get actionable insights to improve the user experience.

Collect contact information from prospects, invitees, and more.

Easily collect and track RSVPs for your next event.

Find out what attendees want so that you can improve your next event.

Uncover insights to boost engagement and drive better results.

Get feedback from your attendees so you can run better meetings.

Use peer feedback to help improve employee performance.

Create better courses and improve teaching methods.

Learn how students rate the course material and its presentation.

Find out what your customers think about your new product ideas.

Resources

Best practices for using surveys and survey data

Our blog about surveys, tips for business, and more.

Tutorials and how to guides for using SurveyMonkey.

How top brands drive growth with SurveyMonkey.

Contact SalesLog in
Contact SalesLog in

Are there statistically significant differences in your data?

surveymonkey-seo-hero

While anyone can see the difference between two numbers, finding out whether that difference is actually statistically significant can take a little more work.

Let’s say you’ve run a customer satisfaction survey for your company, and you’ve got loads of results to analyze. You ask your boss what to analyze first and she says she wants to know if men and women give different responses. For example, do men on average give your company a lower Net Promoter Score℠ (NPS) than women?

When you dig into the data, you see that the average rating from male respondents was 9, compared to an average score of 12 from female respondents. How do you know if 9 is significantly different from 12? That’s where the t-test comes in.

The t-test is a way to determine whether two numbers are significantly different from one another. There are several types of t-test, and each is calculated using a different formula.

1. One-sample t-test: This test looks at whether the mean (aka average) of data from one group (in this case the overall NPS) is different from a value you specify.

Example: Your company’s goal is to have an NPS that’s significantly higher than the industry standard of 5. Your company’s latest survey puts its NPS at 10. Is an NPS of 10 significantly higher than the industry standard of 5?

2. Two-sample t-test: This test examines whether the means of two independent groups are significantly different from one another.

Example: Your hypothesis is that men give your company a lower NPS than women.The average NPS from male respondents is 9, while the average score from women is 12. Is 9 significantly different from 12?

3. Paired t-test: This test is for when you give one group of people the same survey twice. A paired t-test lets you know if the mean changed between the first and second survey.

Example: You surveyed the same group of customers twice: once in April and a second time in May, after they had seen an ad for your company. Did your company’s NPS change after customers saw the ad?

Note that while t-tests can tell you if something is significantly different, it’s up to you to determine whether that difference is meaningful. Small differences can be statistically different if the sample size is large enough.

There are 4 steps to conducting a t-test:

1. Calculate the t-statistic:
Each type of t-test has a different formula for calculating the t-statistic (You can scroll to the bottom of the page to find all three formulas).

2. Calculate the degrees of freedom: 
Degrees of freedom are the number of ways the mean could vary. In this case, the degrees of freedom are the number of NPS ratings that you could have in a given group of respondents. Similar to the t-statistic, the formula for degrees of freedom will vary depending on the type of t-test you perform.

3. Determine the critical value: 
The critical value is the threshold at which the difference between two numbers is considered to be statistically significant.

4. Compare absolute value of the t-statistic to critical value: 
If your t-statistic is larger than your critical value, your difference is significant. If your t-statistic is smaller, then your two numbers are, statistically speaking, indistinguishable.

Let’s walk through the example from the beginning: Your hypothesis is that men give a lower NPS to your company than women. The average NPS from men is 9, while the average score for women is 12. Is 9 significantly different from 12? This is an example of where to use the two-sample t-test.

1. Calculate t-statistic: 

Below is the formula for the two-sample t-test, where:

  • t is the t-statistic
  • x1 is the average NPS for men → 9
  • x2 is the average for women → 12
  • n1 is the number of men who provided a response to the NPS question → say 20 men responded to the survey
  • n2 is the number of women → 23 women responded
  • s1 is the standard deviation of the NPS for men → say the calculated standard deviation is 12.48
  • s2 is the standard deviation of the NPS for women → the calculated standard deviation is 10.51
calculate-t-statistic-t-tests-explained

2. Calculate the degrees of freedom: 

This formula must be used to determine degrees of freedom in two-sample t-tests. The formulas for other types of test are included below.

calculate-degrees-of-freedom-t-tests-explained

3. Determine critical value: 

According to this table, for a two-tailed test with an alpha level of 0.05 at 41 degrees of freedom, the critical value is 2.02. Note that most analysts use a two-tailed test instead of a one-tailed test because it’s more conservative. For more information on the differences between one-tailed and two-tailed tests, check out this video from Khan Academy.

4. Compare absolute value of the t-statistic to critical value: 

Since the absolute value of the t-statistic is 0.86, which is not larger than the critical value of 2.02, then you can conclude that men do not give a significantly lower NPS ratings than women.

You’ll probably be conducting the t-tests in a spreadsheet or statistical program (like Excel or SPSS), but if you’d like to do the math by hand, the formulas for the other two types of t-tests are included below.

t-test-formulas-t-tests-explained

If you decide (as most people do) to conduct t-tests in a spreadsheet or statistical program, the process will be slightly different. Instead of comparing the t-statistic to the critical value, most programs calculate a p-value, which it compares to your alpha level (the most commonly used level is 0.05). In this case, a p-value lower than your alpha level will show that the numbers are significantly different.

Net Promoter Score is a trademark of Bain & Company, Inc., Satmetrix Systems, Inc. and F. Reichheld.

Toolkits directory

Discover our toolkits, designed to help you leverage feedback in your role or industry.

How to make a quiz online: Examples, tips, and best practices

Make a quiz online by following these 10 steps. Following these tips will take you from scratch to having a fully functioning online quiz.

Close-ended questions, examples, and how to use them

Close-ended questions get specific, concise, and targeted responses that allow us to conduct statistical analysis. Here’s everything you need to know about them.

90-day onboarding survey

Ensure your employees are prepared for long-term success with our 90-day onboarding survey template.