Understand Margin of Error and What it Means for your Survey Data
Will your survey results ever perfectly match the population you’re studying? Probably not.
But you can get a good idea of how close you are by using a margin of error calculator. This handy tool will tell you if the amount of people you’re surveying is enough for you to feel confident about the accuracy of the data you collect.
Calculate Your Margin of Error:
|The total number of people whose opinion or behavior your sample will represent.Population Size:|
|The probability that your sample accurately reflects the attitudes of your population. The industry standard is 95%.Confidence Level (%):|
|The number of people who took your survey.Sample Size:|
|Calculate||Margin of Error:—|
What is margin of error in a survey?
Margin of error, also called confidence interval, tells you how much you can expect your survey results to reflect the views from the overall population. Remember that surveying is a balancing act where you use a smaller group (your survey respondents) to represent a much larger one (the target market or total population.)
You can think of margin of error as a way of measuring how effective your survey is. The smaller the margin of error, the more confidence you may have in your results. The bigger the margin of error, the farther they can stray from the views of the total population.
As the name implies, the margin of error is a range of values above and below the actual results from a survey. For example, a 60% “yes” response with a margin of error of 5% means that between 55% and 65% of the general population think that the answer is “yes.”
Let’s see the margin of error formula at work with an example.
How to calculate margin of error in a survey
Imagine you are trying to decide between Name A and Name B for a new product and your target market consists of 400,000 potential customers. This is your total population.
You decide to survey 600 of those potential customers. This is your sample size.
When you get the results, 60% of respondents say they prefer Name A. You need to input a confidence level in the margin of error calculator.
This number expresses how certain you are that the sample accurately reflects the attitudes of the total population. Researchers commonly set it at 90%, 95% or 99%. (Do not confuse confidence level with confidence interval, which is just a synonym for margin of error.)
Try inputting the numbers from this example in the margin of error calculator above. The calculator gives you a margin of error of 4%.
Remember 60% of your respondents chose Name A? This margin of error means that now you know with 95% likelihood that 56% to 64% of the total population–your target market–prefer Name A for your product.
We get to 56 and 64 by adding and subtracting the margin of error from your sample’s response.
How sample size affects margin of error
As we said, knowing your margin of error helps you understand whether the sample size of your survey is appropriate.
If your margin of error looks too big, you will want to increase the size of the sample so that the attitudes of the population surveyed match those of the total population more closely.
What this means is you will need to send your survey to more people.
The Sample Size Calculator can help you easily determine how many people you need to take your survey.
SurveyMonkey Audience makes it simple to send your survey to larger groups of respondents. Choose which specific demographics you want to reach and send them your survey. We have millions of qualified respondents ready to take your surveys.
5 steps to increase your data’s reliability
Now that you know how margin of error is calculated and how it affects your results, let’s review the steps that you need to follow to use these concepts in your survey design.
You can also find a more detailed explanation in this article on how to estimate your population.
Step 1: Define your total population
This is the entire set of people you want to study with your survey, the 400,000 potential customers from our previous example.
Step 2: Decide what level of accuracy you’re aiming for
You need to decide how much of a risk you’re willing to take that your results will differ from the attitudes of the whole target market. This means measuring the margin of error and confidence level for your sample.
Step 3: Define the sample size
Balancing the confidence level you want to have and the margin of error you find acceptable, your next decision is how many respondents you will need. And don’t forget that not everyone who receives the survey will respond: Your sample size is the number of completed responses you get.
Step 4: Calculate your response rate
This is the percentage of actual respondents among those who received your survey. Make an educated guess. If you’re sampling a random population, a conservative guess is about 10% to 15% will complete the survey. Look at your past surveys to check what your usual rate is.
Step 5: You’ve arrived at the total number of people to survey
Once you know the percentage from Step 4, you know how many people you need to send the survey to so as to get enough completed responses.
As we’ve seen, knowing your margin of error (and all related concepts like sample size and confidence level) is an important part in the balancing act of designing a survey. Being able to calculate it will allow you to proceed on sure footing.