Bias is one of the most threatening factors in market research. After all, if the data can't be trusted, what good are the results?
A common bias in market research online surveys or interviews is a leading question. A leading question is worded in such a way that the respondent is influenced to answer the question unnaturally. Though there are other errors to look out for in a survey, a leading question has immediate notable consequences.
A recent read of the Quirk's article, "Small changes can make a big difference," opened my eyes to the underlying issues with leading questions and how to prevent them when writing a survey or discussion guide.
Here are the takeaways I found most important from the article.
It is critical to make sure your questions do not sway the respondent one way or the other.
Balance is a best practice for reliable data.
Why is a leading question problematic?
The article reminds us that availability heuristic at the root of leading questions. In behavioral science, heuristics are mental shortcuts that guide human behavior and decisions. The availability heuristic in particular is a shortcut based on ideas that are top of mind.
Essentially, just by consciously thinking about something will make it appear more important in the human mind. This bias may come in the form of being more accepting to an idea or believing an event happens more frequently than it really does.
The issue here is that so much of market research is driven by asking a question and receiving an answer. If the questions we pose are putting ideas in the heads of the respondents, we have a responsibility to make this wording as objective as possible.
It can take as little as one word to influence the respondent.
One of the best examples of a leading question in the article focused on the words "happy" and "unhappy." A psychology study in 1993 asked half of the respondents "Are you happy with your social life?" and the other half "Are you unhappy with your social life?"
Just by framing the question differently with one word, respondents who answered the latter were almost 4 times as likely to declare themselves dissatisfied or unhappy with their social lives.
Another shocking example of leading questions comes from a 2013 study that asked respondents about their future lives and investments. The question was either asked as how likely they would live to be 85 or older, or how likely they would be to die by 85 or younger.
Since most respondents did not have a preconceived idea of how long they expect to live, the wording variation influenced their answers. Instead of the percentages being mirror images of each other, 52% believed they were likely to live to 85 or older and 70% believed they were likely to die by 85 or younger.
The research then evaluated the respondents' interest in a life annuity. The half who were asked if they would live to 85 or longer were more likely to have intent to purchase a life annuity (39%) than those who were asked if they would live to 85 or shorter (26%). This shows that asking the question with slightly different wording can impact subsequent questions, as well.
The solution? Add balance to the question.
Posing questions that make the respondent think are inevitable in market research. In fact, influencing the respondent to rack his or her brain is often the intention. The key is to probe about an idea in a neutral way.
The article explains that one of the best strategies to avoid bias is balancing a question with a suggestion and its opposite. Instead of asking "why?", ask "why or why not?" The goal here is to make the respondent contemplate both possible outcomes. This helps prevent focus on one outcome that might hinder rational thought.
When it comes to writing an interview guide or designing a survey, always try to read through the questions with the availability heuristic in mind. Just giving it that extra check for bias can go a long way to promote original, truthful answers.
Contact Our Market Research Company
Drive Research is a market research company in NY. We work with a number of clients in a variety of industries to execute market research projects. Our team assists with setting up objectives, defining a methodology, executing fieldwork, analysis, and reporting.
Questions about an upcoming project? Contact us below.
① Email us at [email protected]
② Call us at 315-303-2040