Hello, ¡Hola!, Bonjour, and Hallo, dear survey fans! As promised in Helga’s blog post introducing our new Question Banks in Spanish, French, and German–our post today will give you a glimpse of the science at work behind these exciting new features, and why we think it’s the best thing since sliced bread–or hrm, tortillas/croissants/rye??
The creation process had two main parts: (1) question creation and (2) response options creation.
How did we create the questions?
The process: We partnered with an amazing team of linguists—Daisy, Joseph and Clint–at the University of Florida. They used cutting edge research techniques to build us a computer algorithm that clustered the questions into topics that SurveyMonkey users in French, German, and Spanish care about the most. While the computer algorithm was hard at work clustering, we sent a fantastic group of interns who were native speakers of either French, German, or Spanish to our own methodological boot camp. These interns then created methodologically sound questions that fit into the computer-generated categories. All questions were edited and certified by one of our staff methodologists.
The rationale: When we were developing our approach to International Question Bank, the first question we had to address is: “Why not just translate English Question Bank into another language?” We chose not to take this approach because it’s not just about linguistic differences—there are cultural differences to think about as well. For we methodology monkeys, the importance of the way we created International Question Bank really hit home on the day we spent checking our interns’ “workplace satisfaction” category.
As I bounced from group to group listening to the questions they had written, I was struck by the differences that emerged on a seemingly one-dimensional category. There were similarities to be sure but, the French set of questions focused on office environment, the German questions on office efficiency, and the Spanish questions focused on interruptions in power and Internet service. What these three different question sets had captured is the different ideas of what it means to be satisfied with your workplace in these three different countries (France, Germany, & Mexico) where the initial questions had come from. Even when talking about the same topic, different cultures have different focuses and values. If we just translated our English Question Bank we would have missed these key cultural differences.
The end result: Hundreds of methodologically sound questions that are relevant and meaningful not only for each category, but also for each culture.
How did we create the response options?
The process: The point of using a standard set of response options is that they are evenly spaced along a scale. These standard response option sets exist in English but not–as far as we could find out–in other languages. So, we asked speakers of German, French, and Spanish to rate a variety of words that might be good for response options on a scale from 0 to 100. For example, we would ask them to rate what “often” was on a scale from 0 to 100 if 0 was the same as “never” and 100 was the same as “all the time.” We took all of these individual ratings for each word and averaged them. Then, having defined 0 and 100, we picked the words that were evenly spaced out across the rest of the scale (those words closest to the values of 25, 50, 75).
The rationale: You might think that response options, unlike questions, could be directly translated into other languages from English. However, there’s no guarantee that words in English will have the same strength in another language. “Very” could be thought of as 75 by English speakers, but “muy” could be thought of as 85 by Spanish speakers. There’s just no guarantee that different cultures think about words the same way. Conducting the original research ourselves ensured that these different ways of using language wouldn’t get lost in translation.
The end result: Response option scales that are evenly spaced and methodologically sound for each culture and/or country.
Now that we have a Question Bank for Spanish, French and German, we are hard at work on creating two more: Portuguese and Japanese. Be on the lookout for the launch of those two Question Banks later this year…
Have questions or comments, or perhaps stories about your specific culture’s views of the workplace? What language should we make a QuestionBank for next?
Let us know in the comments section below!