What is a Red Herring Question?
One way to separate the true survey respondents from the reward giveaway abusers is to insert a red herring question in your questionnaire.
A red herring question is a quality control measure in a survey by which you place oddball questions within a series of regular questions to easily identify those who fully read and engaged in the survey and those who are not.
At the conclusion of fieldwork, you can filter your data to show the respondents who need to be removed from the final analysis.
What are the Benefits of Using Red Herring Questions?
Improve data quality
Remember, data is only valuable if it is deemed high-quality. This includes several different factors such as quality of responses and respondent engagement.
Utilizing red herrings are one-way market researchers can remove low quality survey responses or respondents who are not taking the survey seriously.
Flag speedy survey takers
While our market research company always prioritizes the respondent experience by keeping questionnaires on the shorter side, sometimes longer form surveys are unavoidable.
To ensure respondents are answering honestly and accurately, asking a red herring in a longer survey can help keep data clear of survey speeders.
The feedback from speedy survey takers should be removed from your analysis because it does not provide the quality and accurate data you want to rely on.
Subtly catch dishonest answers
Using red herrings in an online survey is a subtle way to maintain the quality of your data. Our experts prefer to use a legitimate question with most of the answers being valid and one wrong answer.
The benefit here is that you are not insulting the respondent’s intelligence or wasting their time by asking a real survey question.
While in the analysis process, our team is able to use valid responses and toss out respondents who fell for the red herring.
3 Sample Red Herring Questions
Tired of reading? In this video experts from our market research company will share what they think is the best red herring question to include in a survey and why.
Sample Question #1: Include outlier answers
One of the best ways to go about a red herring question is to include outlier answers.
For example, if a survey respondent's favorite color is "giraffe", they are clearly not paying attention and just trying to get to the end of the survey.
An example of using outlier answers might look something like...
Which of the following is not a sport?
- Soccer
- Basketball
- Cookies
- Baseball
If the respondent selects something other than "cookies," they're clearly not paying attention.
Using this type of red herring question can help minimize responses from those who are just trying to get to the end of the survey for a reward (or just to be done with it).
Include a forced response within grid questions
Many red herring questions occur in Likert scale grid questions where respondents are asked to rate a number of factors using a 1 to 5 scale or a 0 to 10 scale.
In these grid questions, the red herring question will often ask the respondent to select a specific number (e.g., select 4 or select 2) within the grid series.
Here's a real example of a red herring used grid question by Drive Research.
As you can see, the question is phrased at the top. Each row represents separate behaviors and each column represents the different response options for each behavior.
Note, the second to last row states, "Select "no change" for this category. This is where the red herring comes into play.
Anyone who does not answer accordingly will be removed from the analysis.
Something to keep in mind…
Grid questions are one of those things in market research that are great when using sparingly. Typically, when Drive Research designs a survey for our clients, we like to use grid questions that include up to 10 rows maximum - but sometimes it is necessary to go beyond this.
Additionally, our online survey company likes to limit the number of grid questions used in a survey because it can be really cumbersome to respondents.
Again, sometimes it's necessary to include multiple grid questions to meet the goals and objectives of the survey.
Include a fake brand or company name
This red herring question works by including a strange company name among a list of well-known brand names as answer options.
The idea here is that respondents who are not paying attention may select the fake company name and be flagged for poor quality removal later on.
Here is an example of using fake brand names.
Which of the following phone manufacturers have you purchased from? Select all that apply.
- Apple
- Samsung
- Motorola
- Phony
The idea here is that any respondent who selected the fake phone manufacturer (ie. Phony) is absolutely not paying attention during the survey. This easily detects which respondents' feedback should be cleared from analysis, including the other answers they provided for the survey.
Are you interested in working with our online survey firm? Having trouble finding survey respondents? Need a third-party to assist with data cleaning?
Drive Research can help! Our experts will consult with you from end-to-end to give you exactly what you need for your market research project.
Contact us through one of the four options below.
- Message us on our website
- Email us at [email protected]
- Call us at 888-725-DATA
- Text us at 315-303-2040
