Channel this patient woman when sending your online survey to get results you can count on.
Getting good quality data from your online survey is not just a function of how you ask your questions and which questions you ask—it’s also about understanding the context of whom you’re sending your survey to, and when you’re sending it.
Let’s look at an example. Before September 11, 2001, President Bush’s approval ratings hovered around 50-60%. After 9/11, his approval ratings shot up to 90%. So if you had sent out a survey on September 10, 2001, the answers would look drastically different from the answers to the same survey you sent out two days later. But what if it’s September 12, 2001 and you want to send your survey? If you send it out right away, you would draw the conclusion that nearly everyone approves President Bush and the job he was doing, even though only half approved the day before. Attitudes can be highly sensitive to context.
Even if your survey has nothing to do with major public events, the people who respond to your survey might give different answers at different times. For example, let’s say you’re interested in general attitudes towards drinking alcohol. If you send your survey on a Friday night, the people who are at home and ready to immediately respond to your survey are probably less likely to be big drinkers than those who are out and unable to fill out your survey. If you wait until Monday morning to send out your survey, the responses will probably look a lot different.
The effect of timing on responses you get is not always obvious as the examples above. Here at SurveyMonkey, we tested the effect that response day had on answers to survey questions. We sent out the same exact survey each day of the week, asking about a wide range of topics, such as food, cars, movies, and vacation. We found that responses changed based on what day the survey was sent out and what day the person responded. For example, people who answered our survey on a Monday reported taking fewer airplane flights than people who answered on Wednesday. That might matter if you’re a travel agent looking to get new clients — but is unlikely to be something you could have predicted.
So what can you do to prevent these problems? Patience.
As your mom probably once said, “Patience is a virtue.” For our purposes, this means you should be patient in terms of choosing the right time to send your survey. So, if a devastating terrorist attack just happened, you might want to wait a few months before you ask questions about the president. It also means that you need to be patient when it comes to how long you’ll want to run your survey for. In the example above about drinking alcohol, the problem could likely be fixed by letting the survey run Friday through Tuesday, giving people with all kinds of different habits a chance to respond. This tactic of letting a survey run for at least five days is a good rule of thumb. It will help with those non-obvious situations too, like the airplane flights example, where you’re not sure why a question gets different answers at different points in the week — it just does.
“Five whole days,” you may be thinking, “I can’t wait that long! This survey is really important!” We know, we know, it’s not any fun to be patient. You want your answers now! In fact, you want them yesterday. But think of it this way: if you’re patient with your survey now, you’ll get great results. Great results can mean that you market your product correctly and more people buy it, or that you get more donors to pledge money to your non-profit. Patience in surveying gives you the peace of mind knowing that you didn’t just get answers — you got the right answers. And that’s the biggest virtue of all. One that your boss (and your mom) will certainly be proud of.
*Image courtesy of Big Stock Photo. Thanks Big Stock!