Just the thought of a survey farm accessing a survey link makes us cringe.
Even worse? If a survey farm learns the screening criteria/target audience and how to pass the data quality checks.
Market research online sampling techniques have progressed so quickly that identifying survey farms can feel more elusive than ever. In fact, there has been increased interest in more traditional survey methodologies (i.e., phone surveys, intercept surveys) due to the presence of survey fraud.
Our third-party survey firm will discuss preventing survey farms from accessing surveys and how to identify them for removal if you suspect a survey farm contributed to your responses.
What Are Survey Farms?
Survey farms are likely exactly what you are thinking… it’s a highly organized, modern process to round up a group of survey respondents, often using incentives.
The incentives for participating in research can vary, but most have some sort of monetary value. For example, a research panel may use points that can be redeemed for products or cash once enough points are accumulated.
Typically, survey farms make numerous phony attempts to complete a survey.
These attempts can be made by a single person, a single bot, several people, or several bots.
The goal is either to complete the same survey as many times as possible or as many different surveys as possible to receive incentives/rewards for completion.
In both cases, the responses should be removed.
Survey Farms Impact on Data Quality
Research findings are based on clean data gathered from engaged respondents, so no researcher wants fake data lurking in their responses.
Finding data from a survey farm can be alarming. These responses should be removed promptly and the sample plan should be adjusted to avoid further responses from the survey farm.
Since survey farms likely contribute several completes to a survey, the removal of these responses will have a large impact on the overall findings.
This is how you can you identify responses from a survey farm:
- The responses may seem eerily similar
- They may come from a similar location and/or the same IP address
- They may start and end the survey around the same time (i.e., submitted in batches)
- They likely took an eerily similar time to complete
- The open-ended responses seem too detailed or give you a sense that it was copied and pasted from a reputable source
Strategies to Prevent Survey Farms
There are multiple ways to prevent survey farms. One of the best ways to prevent this is to partner up with a third party firm like Drive Research. Below, we’ll cover a few other strategies that can prevent survey farms.
Writing Effective Survey Questions
When it comes to writing an effective survey, there’s more than ensuring the survey objectives fully meet the client’s needs and adhere to survey writing best practices.
Put checks in place to ensure high data quality. Some common ways to do this include red herrings, attention checks, and so on.
Additional factors to consider include survey flow and response masking to limit bias and avoid revealing the purpose or screener criteria.
For example, suppose you have a survey that is meant for full-time IT professionals. Asking a simple yes or no question to confirm whether respondents are IT professionals is a big tip off to survey farms about who the target audience for the survey is.
Rather than taking this approach, first confirm employment (i.e., full-time, part-time, self-employed, unemployed and looking for work, unemployed and not looking for work, retired, disabled, other).
Then, ask what their role is grouped with others (i.e., marketing, sales, logistics, engineering). This helps mask/hide the screening criteria and the target audience.
Following Data Cleaning Best Practices
There are a variety of ways to spot poor survey responses during the data-cleaning process.
Take a look:
- Use quality-check/comprehension questions throughout the survey
- Ensure respondents are unique (i.e., no duplicates based on custom data fields like IP addresses)
- Review how long the survey took to complete
- Ensure respondents did not straightline
- Review open-ended questions
- Put checks in place to prohibit bots (i.e., reCaptcha) and traffic from geographies not of interest
Below are the types of low-quality respondents to look for:
- Speeders: Those who answer a survey so quickly it’s likely they were disengaged
- Professionals: Those who complete surveys for financial gain
- Disengaged: Those who don’t read directions/miss data quality check questions
- Tech wiz: Those who try to complete surveys more than once for financial gain (they likely use a VPN, etc.)
Use Trusted Panel Partners
Partnering with an online survey company like Drive Research provides access to countless vetted panel partners.
Find a partner who reassures you that they only work with trusted panels that provide quality respondents and use industry-leading best practices to prevent survey fraud. Doing this will eliminate additional time, stress, and resources.
Use reCaptcha
At the beginning of the survey, implement a reCaptcha to prevent automated responses. It’s also a good idea to consider asking respondents to verify their email address.
These are quick and easy adds to help prevent basic bots from entering a survey.
Contact Our Online Survey Company
With years of combined experience in market research, our team of survey pros will partner with your business to create a project that yields actionable, quality feedback. We’ll be there every step of the way, from our initial kickoff meeting to the final debrief.
To learn more about our market research services, get in touch with us today!
- Message us on our website
- Email us at[email protected]
- Call us at888-725-DATA
- Text us at 315-303-2040
Emily Taylor
As Director of Operations, Emily does more than wrangle data. Her work includes executing company OKRs, company-wide project management, training/onboarding, team culture initiatives, and more!
Learn more about Emily, here.
