Online Surveys = Ongoing Insights

Presidential Poll Tracker, SurveyMonkey Style

Presidential Poll Tracker, SurveyMonkey Style
Screen shot 2012-10-24 at 12.44.28 PM

Picture this. You’re curled up in bed watching the nightly news…“Obama is leading in the polls!” says a pundit decisively. When a commercial comes on, you change the channel. “Romney has a massive lead in the polls!” says another pundit with confidence. Perplexed, you change the channel again. “Obama and Romney are dead even!” proclaims yet another pundit.

What’s going on?

Why are poll numbers so different? Is it because the race is very close? Or because there’s something wrong with the way polls are measuring people’s opinions?

Well, let’s take a step back and look at how polling works.

First, polling companies call up U.S. residents on their landlines or cell phones. Of these people, only 9% on average, agree to participate in the phone poll.

Why do we care about response rate?

Well, who are these 9% of people who are agreeing to do a cell phone survey? And are they really a representative sample of the national population? Maybe. If they aren’t, why is this a problem? Well, if you’re not getting a representative sample of your population of voters, your prediction won’t be correct on Election Day.

Let’s look at a historical example of this very problem.

For the 1948 presidential race, the Reader’s Digest predicted that Dewey would beat Truman. What happened? Well, the Reader’s Digest sent their survey to all their subscribers by mail, figuring that they had enough subscribers to make a good prediction. The problem was that most of their subscribers supported Dewey, whereas the rest of the country was much less enthusiastic about Dewey. An online survey might make a similar mistake by assuming that since lots of people use the Internet, its results are fine. In reality, it might only reflect the opinions of rich fashionistas who were emailed the survey link through a Chanel mailing list.

So how does a phone poll work around this problem?

First, phone polls have the advantage of being able to call anyone in the 95% of the U.S. population who has a phone. This means that their sampling frame is nearly identical to the population. Phone polling firms also frequently weight their data to correct for any bias in their sample. Each polling firm, however, has its own weighting mechanism, and these can produce variant outcomes.

Another possible solution to this problem?

Find a better way to reach more people! Enter, the internet and online surveys. Roughly 80% of households have internet access, and roughly 90% of workplaces do—and these numbers are growing everyday. Moreover, at SurveyMonkey, the average response rate to our online surveys is nearly triple that of the phone poll. So, online surveys can reach an only slightly smaller population of US voters, three times more quickly!

A unique opportunity.

Every day, more than a million people in the United States access SurveyMonkey to fill out a survey. This heavy volume allows SurveyMonkey to measure day-to-day shifts in public opinion in a way that phone polls cannot. This means that a daily average will be strongly linked to whatever political events occurred the day before. Additionally, the topics of SurveyMonkey surveys vary widely, from market research to political surveys to party planning to academic experiments. This means that not only do many people take a SurveyMonkey survey every day—many different kinds of people take a SurveyMonkey survey every day! Consequently, we can be confident that we’re not just getting one particular kind of person, but a snapshot of the entire country.

So, what do we have here?

This week, we’re unveiling our effort to estimate voting preferences in the upcoming election. In a polling field cluttered with phone polls, we are one of the only online polls to attempt this. We’ve collected nearly a million responses over the past two months and, so far, the data closely mirror the publicly available averages of phone polls.

Here are some highlights of our results so far…

  • Survey response rate: 30%
  • Sample: 810,477 registered, likely voters
  • Compared to phone polls: SurveyMonkey’s data is more volatile. This means that the numbers have gone both up and down over the past two months more often than the numbers in phone polls have. This means that SurveyMonkey’s data can show the impact of events like Romney’s RNC speech, Obama’s DNC speech, the leaked 47% video, and the debates on voter preference. This impact is much more difficult to see in the phone polls. To smooth our data and make it a little less sensitive, we use a 7-day average (that is averaging the numbers we get over the past week). This makes our data look very close to the phone poll data. (This makes sense, since phone polls all average their data over the course of anywhere between 2-12 days depending on the poll.)
  • SurveyMonkey poll (as of 10/19): Obama, 47.1%, Romney, 45.2%
  • Real Clear Politics average (as of 10/19): Obama, 47.1%, Romney, 47.0%

Want more?

  • Check out the all the details of our results so far in the PDF embedded below. Or, click here to access.
  • Stay tuned to our blog for up to the day updates on the presidential race…

Tags: , , , , , , ,

  • Catie

    Super interesting way to look at things. I like the idea of having results so quickly- seeing fluctuations from things like debates, unemployment % announcements, etc.

    • Sheila G

      Thanks Catie. We think so too. Stay tuned to the blog for more results from our presidential poll tracker!


    good……… is the best for survey monkey

  • Phuti Mongalo

    I really like survey monkey’s quick analysis.

    Phuti (RSA)

  • gona srinivas


  • H. B. Koralegedara

    I think Mr. Obama should be re- elected as he is one of the moderate prsidents of USA as i have seen.

  • Taremwa Brian

    I support OBAMA Barrack

  • I pray to GOD that Barak Obama wins.. he’s a more human being than Mitt.
    I never thought there will be a day when I support democrats, well, seems this day has finally came…

  • Gyles Glover

    Great analysis. Have you tried to simulate results in the state-based electoral college structure? Looks like you should have the data to do this.

  • Samuel Dada

    The biasness in the opinion poll shows lack of representation of the population as you earlier mention, the goodness of the volatility in SurveyMonkey data captured the undecided voters who can cause shock in the outcome of the election.

  • MOLZ

    COOLIES!!!!!!!!!!!!!!! 😀

  • Bmaier

    Telephone polling is heavily weighted to land lines in most states. The younger the population the less likely they will have land lines and only use cell phones. So polling is skewed towards older generations as well who may be perceived as leaning towards republican. I like your methodology.

    • Susan Frierson

      true…i don’t know anyone under 35 with a landline anymore….

    • Jessica

      Bmair, I’m with you. I used to work in college for the psychology research department doing telephone surveys. The only people who would actually agree to complete a phone survey were either elderly or just plain crazy! Definitely not representative of the population… Plus, people don’t always give their true opinion when talking with a live person, as opposed to the internet where they can remain anonymous.

      • Valerie

        You are correct, but believe me, it’s not just telephones, it’s computer generated e-mail, snail mail door hangers and people actally going door to door. Belive me I’ve received them all several times over. Let’s not forget all the tv ads, even on cable stations.

  • Bill C

    Really can’t dig into the actual answers to the questions. example: how important is the presidential election? secondly: there are two anomalies that are inexplicable. after the first debate there is immediate reaction that follows the national trends but in the days that follow, there is a sharp reversal with no intervening event. that reversal simply didn’t exist in the anywhere in the national polling. this casts doubt on the entire results.

  • Peg

    Sorry, but I like this poll better!

    •Real Clear Politics average (as of 10/19): Obama, 47.1%, Romney, 47.0%

  • taras smetaniuk

    If the electorate gets the government they deserve they will get Romney if they elect the government they need they will re-elect President Obama

  • Doldrums

    Polls are useless if the don’t take into account the Electoral College. You can win the general election and still lose the race.

  • Marissa

    Love that you did this and shared results – so interesting! I love that you took your product to new heights – good job!

  • Polling over the phone is very difficult nowadays because not every one has a land line. I know that I used to be polled, but now, because I only have a cell phone, and the area code doesn’t even reflect where I live, I’m not being polled. And it’s the same for many other people- both retired and young adults!

    Perhaps it would be interesting to find out the demographics of people who only have cell phones: are there more Republicans than Democrats? If so, you are not getting a clear idea of who you are actually polling, and it can skew the results.

  • Concerned Customer

    How did you get the names and email addresses of those you survey for this? Are you using our private contact lists?

    • Sheila G

      Hi there. Good questions! The privacy and security of our customers is of the utmost importance to us and therefore we did not collect names or email address for this project. Every day, more than a million people in the United States access SurveyMonkey to fill out a survey. We’ve used a portion of that traffic and invited survey respondents to complete our election survey after they completed the initial survey they were sent. There’s no need to sign up, register, or provide contact information. We’re simply gathering feedback from voters interested in sharing their opinions.

  • Russell Padgett

    What thoughts do you have on SM’s audiance being scued by the demographics of the user base. That is to say, it seems that the users of SM would fall into some strata of demographic be it professional, wealth, etc and would not be well reflective of the entire voting population. thoughts?

  • I’m increasingly feeling that polls using phone calls (either to cell or landline) reflect a small minority of people who 1) answer their phone 2) don’t have caller ID which immediately skews the results. Of course, any methodology has its own skew, but I’ve been wondering how the digital crowd would view the election and what the impacts of particular events really were. Thanks for taking on this initiative. Its very interesting.

  • John Andrews

    Oh, no!!! Another misleading poll that tries to force voters to pick between the two big money candidates. Surely you can do better than this. If you can’t include candidates like Jill Stein and Gary Johnson, you should just get out of the polling business. Trying to narrow voter options is despicable.

  • Judy

    So, now that it’s all history, how did your latest poll do?

  • Every participant observed advantages throughout the critique.

Inspired? Create your own survey.

Inspired? Create your own survey.

PRO Sign Up Sign Up FREE or Sign in

Write Surveys Like a Pro

Write Surveys Like a Pro

Ever wonder what SurveyMonkey’s really made of?

Ever wonder what SurveyMonkey's really made of?

Read our engineering blog »