When it comes to bias in market research, it's hard to keep track of the number of variables and influencers which could impact your results positively or negatively. Bias wears many faces. It's everywhere. Bias finds it way into every step of the market research process.
As a market research consultant you need to identify and neutralize bias as much as possible. Identifying bias is something that you learn and getting better with over time. As your years in market research go by, identifying bias will become second nature to you. It's one of those things that once you are aware of, you don't even think twice about identifying and eliminating in your projects.
Where does bias live in the market research process?
As I mentioned before, bias can be identified in all phases of the market research process. There is methodology bias in that a market research may favor specific types of projects (e.g., traditional focus groups or in-depth interviews) even though there is a better and more suitable methodology for the client's objectives (e.g. online focus groups).
There is a lot of bias that crops up in survey design from wording, to ordering, to sequence. Survey writing needs to be a scientific process which produces the most unbiased and accurate results possible.
Amateur survey writers and agencies who do not specialize in market research but offer surveys and focus groups do more harm for your organization than good. They likely have not been trained, certified, or educated in market research. Using these types of surveys which often produce the wrong results and a poor path is worse than not doing market research at all.
Bias in market research also pops up in fieldwork. Whether it is interviewer bias or acquiescence bias, the moderator conducting the market research can have a large impact on results. If the interviewer is overly nice, it could influence the interviewee to offer more positive feedback. Several other forms of person-to-person bias exist as well. This bias can also creep into the analysis, interpretation, and reporting for the client.
Suffice to say, bias is a scary thing. If your market research consultant isn't aware of these through training, certifications, and past experience, it could cause more problems than solutions for your organization. This area is where "do-it-all" agencies typically fall short when they make the claim to also specialize in market research.
Market research is a niche expertise, and it needs to be treated that way
All companies cannot be good at everything. As a decision-maker you should be wary of those who claim to offer all things marketing including market research. Market research is much different. It's a niche topic that requires specialties and training to create a process which is reliable, accurate, and actionable.
Just like you would not hire a veterinarian for your own personal appendectomy. Could he or she do it? Possibly. Would you trust it? Much like veterinary surgery, market research is a whole different animal - literally and figuratively. It requires different experience, different training, and different skill sets in order to be done well. If done incorrectly, there are significant negative consequences.
Could she operate on you? Possibly, but would you trust it? When it comes to market research it's either all you do, or you cannot do it well. It's a niche service and one that requires a lot of time and attention to do well.
To dive a little deeper in today's blog post we are going to take a closer look at a specific type of bias and what you can do to eliminate it. This bias crops up in the survey design portion of your market research project. It often goes by the name of sequence or order bias.
What is order bias?
Order bias is a term used in market research which speaks to how respondents are influenced to perceive or select specific responses based on the order they are shown in the survey. There is something called "first-shown" bias which means respondents are more likely to select the first item in an list than others in that same list. When designing a survey, all precautions must be taken to ensure the items you test in your market research project are backed with valid and accurate data.
What are some common options to eliminate order bias?
The great thing about this type of bias (if there is anything at all) is it is fairly well-known. Therefore many online survey platforms (if not all at this point) have built-in mechanisms to help the survey writer eliminate this problem. When writing each survey question, options exist to address order bias and here are a few examples of what you can choose from.
Randomize
This is likely the most common way to eliminate sequence or order bias. By selecting randomize the radio button series shown to each new respondent will be ordered randomly. For example, let's say you ask, "Which social media platform do you use the most?" You ask them to select from a list of LinkedIn, Facebook, Instagram, Twitter, or Snapchat.
Based on past data and known circumstances, market research and the "first-shown" bias teaches us that LinkedIn will be overstated a bit because it would be first in the list. By randomizing these choices it eliminates this bias and shows the 5 social media platforms in a random sequence each time to each new respondent answering the online survey.
The second respondent may see a list like Twitter, Facebook, LinkedIn, Snapchat, or Instagram. The third respondent would see Facebook, Snapchat, LinkedIn, Twitter, or Instagram. You get the point. Randomization ensures unbiased responses.
Flip
Another common way to eliminate order bias in market research is to flip the answer choices. This is often called "inverse" on some survey platforms. What this means is that for each new survey respondent the order of answer choices will be flipped. Using the same example from above, the first respondent will select from LinkedIn, Facebook, Instagram, Twitter, or Snapchat. The next respondent would select from Snapchat, Twitter, Instagram, Facebook, or LinkedIn. The third respondent would then revert back to the list that begins with LinkedIn.
In Conclusion
Make sure you work with an experienced market research consultant who understands all forms of bias in the process and how to mitigate it. When the market research consultant understands things like order bias it gives the market research greater accuracy, reliability, and trustworthiness. Eliminating order bias is a basic example and a way to separate the amateur agencies who dabble in market research as a side business from the experts and thought-leaders in the industry who solely live and breath market research day-in and day-out.
Looking for more information on survey writing? Here are 5 tips to write a better survey.
Drive Research is a market research consultant in Syracuse, NY. Interested in learning more about our company and the market research services we offer?
Call us at 315-303-2040 or send us an email at [email protected].