One of the most common questions that is posed to our market research company on a regular basis is: "How many survey completes do we need?" The answer is never a clear cut and dry one and it depends on a number of factors that I'll discuss in this post.
The one myth when it comes to surveys is a 100% response rate. Oftentimes clients without any experience in market research might assume if we send 100 emails, we'll get 100 to respond right?
This is never the case unfortunately. Respondents are busy, do not participate in surveys, do not want to share feedback, and for a variety of other reasons just simply won't participate. We're still searching for the 100% response rate on a regular basis at our market research company. Believe you and me, that's our goal for every single project that comes through our door.
So now that this is settled, let's discuss some factors that impact exactly how many survey responses you need from your online survey, phone survey, mail survey, or intercept survey market research project.
How many survey completes do you need? It depends on the following factors.
How Large is Your Sample Pool?
Is the audience you are surveying 100 respondents or 100,000 respondents? This will impact the number of survey completes you should obtain. In cases where your sample pool is 1,000 or more respondents, as a standard we recommend a minimum of 100 survey completes. With a minimum of 100 surveys completes, your results will carry some statistical reliability and you can trust the data to be directional.
When we say directional, we mean although the results may not say turn left at exactly 38 degrees, it will say turn left. Just knowing what direction to head with marketing or strategy or which tactic to choose is enough guidance. Oftentimes this high-level outcome is what our clients are searching for with our market research company.
What Degree of Statistical Reliability Are You Looking For?
After you know the sample pool available to you, the next step is to determine the degree of statistical reliability you are looking for. Knowing you will not obtain a 100% response rate and the difficultly of obtaining a large number of completes from a small sample pool, the next step is to look at a degree of confidence you are aiming for.
As a general guide, margin of error is the range of error for each percentage or data point in the research. If your margin of error is 5% at the 95% confidence interval (most common), this means if the survey were conducted again with another random sample, results would yield within +5% or -5% of each percentage.
So, if your customer satisfaction is 80%, the market research is 95% confident that it would range between 75% and 85% if the study were conducted again.
As a general guideline, here are the approximate margin of errors for several common levels of survey completions:
100 completed surveys: +/- ~10%
200 completed surveys: +/- ~ 7 to 8%
400 completed surveys: +/- ~4 to 5%
1,000 completed surveys: +/- 2 to 3%
The more data, the better. The more you have, the more reliable the data becomes. However, with more data typically comes increased costs. So choose a level you are most comfortable with from a statistical reliability and budgeting standpoint.
What Segments or Cross-Tabs Do You Want to Analyze?
The other layer of consideration you'll need to review is the number of cross-tabs you want to include. If you choose to conduct 100 surveys and want to break results down by age groups by gender, the reliability of each age group's and gender results is less valid. Reviewing 50 completed surveys of those under 45 against 50 completed surveys of those 45 and over increases the margin of error of each because the base is no longer 100 completed surveys. Now cut the under 45 age group by male and female and you can see where the sample size continually decreases.
Knowing how you want to slice and dice the data going into the market research will help you understand the levels of completes you need to obtain to make each sub-segment reliable. If you want to cut the data by 4 separate geographies you should look to obtain at least 50 to 100 completes or more for each segment.
Contact Our Market Research Company
Drive Research is a market research company in New York. Interested in conducting an online survey, phone survey, intercept survey, or mail survey with our market research company?
Contact our team at [email protected] or call us at 315-303-2040.