Online Surveys = Ongoing Insights

Why Sampling Matters: The Secret to Success in Our Election Day Predictions

 → 
 → 
Why Sampling Matters: The Secret to Success in Our Election Day Predictions

This year we reprised our election project to predict the outcomes of the Virginia and New Jersey gubernatorial elections. We are proud to report that we correctly predicted the winners, and were within a few percentage points of the actual vote totals.

And the best part is we did all of this using UNWEIGHTED, RAW data. How did we do so well? Part of the key to our success was the high quality of our sampling process.

You’ve probably heard that “sampling” is something that you should do when sending out a survey. But what exactly that might mean or why you should care might be a little fuzzy.

Well, we’re here to illustrate how vitally important good sampling technique is to the success of your survey using some data from our 2013 election project. Our take-home point is that how you hunt down the people you ask really does matter. It’s not just what you ask and how you ask it, but WHO you’re asking!

In sampling you have 3 important factors to consider:

  • Population:                 The people you WANT to contact
  • Sampling frame:          The people you CAN contact
  • Sample:                       The people you DO contact

Sampling frame_1

In the context of our election project here is how those 3 factors play out:

  • Population:                 Eligible voters who show up to vote in a state
  • Sampling frame:          Anyone in that state who takes any SurveyMonkey survey
  • Sample:                       Any of those survey takers who take our election survey

Here’s some maps of those same sampling variables from our 2013 Election Project:

Governor Race

(A darker color means more people came from that part of the state, a lighter color means fewer did.)

So, why is sampling so important? Well, if we have a sampling frame that is representative of our population, this gives us more leeway with our actual sample. If we don’t, then we’ll have to work harder to make sure the sample is representative of our population.

The high quality of our sampling frame in our election project (indicating that it does a good job of representing the population) means that our actual sample can deviate a little more from the actual population without it being a problem. When you have a more representative pool (in this case, anyone completing a SurveyMonkey survey) to draw people from than say, people who pick up their phones during dinner, your chances of uncovering the truth are a lot higher.

Well, there you have it! Now you know why sampling matters AND the secret to our success in the 2013 Election Project. Smart sampling.

So, on your next survey, before you hit send – think about WHO you’re sending it to. It’s the best way to make sure you get not just answers, but the RIGHT answers.

Tags: , , , , , , , , , ,

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in
Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »