Behold, one of the major red flags in market research: survey straightlining. 🚩
Caught during the cleaning process after survey fieldwork wraps, straightlining can have a significant negative impact on the data. Thankfully, it's pretty simple to catch from the researcher's perspective.
In this post, our online survey company will cover what straightlining is, how it impacts survey data, how it's detected, and more. Let's go!
What Is Survey Straightlining?
Straightlining is the act of selecting the same response repeatedly down a line of answers on a survey. Typically, straightlining is most common in grid rating questions.
Grid rating questions are often shown in a table format where numerical scores are shown along the top (e.g. 0 to 10) and factors to rate are shown down the side of the question (cost, time, customer service, etc.) Straightlining would be the act of selecting all "10s" or all "3s."
Usually, straightlining is a dead giveaway that the respondent is:
- Uninterested in the survey
- In a time crunch
- Currently lacking the mental energy to take a survey
Ultimately, these types of responses can have a major impact on the quality of data retrieved.
How Survey Straightlining Impacts Data Quality
When a respondent chooses the same answer over and over again, no thought is going into the process. This means that their responses aren't truly reflecting their opinions; and if respondent opinions aren't authentic, then the survey data isn't either.
If these inauthentic responses are left in the data, this can skew the final results delivered to the client. Because of this, it will be difficult for them to make effective data-driven decisions based on the feedback.
Ultimately, survey straightlining is a waste of resources for both the researchers and respondents. On the research end, it negates the effort put into creating a quality online survey. For the respondent, it simply becomes a waste of time!
Ways to Check for Straightlining in Surveys
Researchers are well-trained to keep an eye out for straightlining during the data cleaning process. Below, we'll cover some common ways straightlining is spotted.
Analyze Response Patterns
Survey analysis software is especially helpful in detecting survey straightlining.
Specific steps analysis software completes include exporting the data into a spreadsheet and sorting responses for grid questions.
Both of these actions allow the researcher to visually inspect the data for straightlining.
Using these tools will not only help researchers visualize response patterns, but they will determine the frequency of straightlining as well.
Set Thresholds
Setting a fixed number or percentage of identical responses to flag a respondent is another useful action to take.
For example, if a respondent selects the same option for 80% or more of the items in a grid, they can be flagged for potential straightlining. This will alert researchers to keep a closer eye on the responses from the specific participant.
Manual Review
Then, there's the classic manual review of survey data.
In this case, researchers will review flagged responses to confirm straightlining. They will carefully consider the context of the survey, along with the specific respondent's pattern of answers.
Since researchers are well-trained in spotting straightlining, it's likely they will pick up on it rather quickly during a manual review.
Recommended Reading: Online Survey Testing - What to Look For Before Hitting Send
How to Avoid It
The good news? Straightlining can be avoided in many cases.
Avoiding Certain Question Styles
This includes both grid and matrix questions. These types of questions can trigger poor responses, upping the chance for survey straightlining. Respondents will be less likely to do this if there is more of a variety of questions that require thoughtful responses.
Offer One Question Per Page
More often or not, presenting a single question on a page reduces the chance of straightlining.
Responses Are Directly Linked to Question
Meaning, when a number of questions are placed on the same rating scale, respondents are more likely to straightline. By including response options that are more relevant, respondents are prompted to put more thought into their answers.
What To Do If You Suspect Straightlining
If you notice straightlining, all is not lost (yet)!
Before completely discarding the data, use other factors such as open-ended comments and the time it took to complete the survey. These factors should be reviewed to make an educated decision on whether or not to remove the survey case.
If you notice the respondent skipped the open-ended questions and the time to complete the survey was 2 minutes compared to the average of 9 minutes, you can then remove the case.
Contact Our Online Survey Company
It's true: researchers hate straightlining. Determining when this occurs is key in the data cleaning process, as straightlining can greatly impact final analysis.
At Drive Research, our team of online survey pros boast years of combined experience.
Because of this, we're experts at delivering only the highest quality survey data to clients.
To learn more about our market research services, reach out through any of the four ways below.
- Message us on our website
- Email us at [email protected]
- Call us at 888-725-DATA
- Text us at 315-303-2040
George Kuhn
George is the Owner & President of Drive Research. He has consulted for hundreds of regional, national, and global organizations over the past 15 years. He is a CX-certified VoC professional with a focus on innovation and new product management.
Learn more about George, here.
