In the most recent addition of Quirk's magazine, I came across an article that discussed the impact of various variables on survey response rates. This article was written by Lori Dockery from Vernon Research Group. The article summarizes a study put in place to understand the impacts of several variables on the likelihood of respondents to complete a survey.
Working in market research for 12+ years, this has always been a hot button topic among our colleagues and clients. It's a constant top-of-mind topic whenever a proposal is being drafted or a survey is being constructed by our team.
- If we reduce the reward by $25 how will that impact our response?
- If we add 3 questions to the survey will we see a lot more drop out?
- Can we swap out a reward payout for a raffle?
- Can we get away with asking 1 more open-ended question?
These are common questions with both the market research team and the client each and every day. Well, this author attempted to arrive at the answers to these using over 100 unique data points combined over the past 4 years.
Variables examined were B2B and B2C respondents, local versus non-local respondents, incentives (gift card, points, free report), value of incentive, types of analysis or questions in the survey (conjoint), number of ads, numeric and open-ended boxes, whether the study was blinded, and completes needed among others.
It's research on research, and we loved it.
What impacts survey response rates?
This author really dove into the data at a microscopic level.
Here are a handful of our key takeaways on what impacts survey response rates.
Takeaway 1: Completion Time Plays a Major Role
Not too surprising here. The longer the survey, the more chance you'll have of respondents dropping-out. When it comes to online surveys, the shorter the better if you need a lot of responses and a lot of responses fast.
This study in particular used a state-wide panel found the completion rate drop below 90% around 14 minutes. A maximum time to complete of 15 minutes seemed to be a standard for many market research firms. We find in our general population surveys (non-panel) this threshold is significantly lower.
When administering surveys through social media we place our cap on length at 3 to 5 minutes. Extending beyond that significantly increases our drop-out rates. With customer surveys this is more like 5 to 7 minutes maximum. Panels do offer some more flexibility in time because they have a relationship and vested interest.
However, the point remains that long surveys are not recommended. You must keep it under 15 minutes. Best practice is under 10 minutes. But optimal length is under 5 if you are able to cover your core line of questions and objectives in that timeframe.
Takeaway 2: Drawings and Free Reports are Not Strong Giveaways
Although client budgets often dictate levels of rewards and giveaways provided to respondents, the data suggests free reports shared with respondents at the close of fieldwork is not a strong option. These free reports and infographics are often dangled as a carrot to help acquire interest and completion but they are not highly correlated with completion rates.
Wouldn't it be great if we could give everyone hundreds of people who completed a survey $10? Well, we actually just did that at Drive Research which might have led to the 10%+ response rate we received on a blinded mail survey. This isn't always realistic for a client.
However, offering points or an actual value to all survey participants makes a much larger impact on completion rate than a sweepstakes, raffle, or chance to win a reward.
Takeaway 3: Be Careful Getting Too Scientific with Your Survey
Including a series of questions like conjoint or the van Westendorp pricing model provide lots of excellent analysis but they come at a price. These types of exercises can be cumbersome for survey respondents and take a lot of time and energy. Not surprising here that they are negatively correlated to response rates. Meaning the more you use these in your survey, the stronger chance of drop-out. It's a trade off.
Contact Our Market Research Company
Drive Research is a market research company in Syracuse, NY. Learn more about our market research services here including online research options.
Interested in receiving a quote or proposal from Drive Research? Contact us at [email protected] or call us at 315-303-2040.