While you build your survey, you have two survey design options:
- Open-ended questions
- Close-ended questions
An open-ended question is a free text box that allows a survey respondent to type in a response, while a closed-ended question is when a respondent is forced to choose among shown answer options.
Both have their own pros and cons for both the market research firm and the survey respondent.
The good news is you only have 2 options. The bad news is you only have 2 options.
In this post, our market research company covers the pros and cons of open-ended and closed-ended questions for both the survey writer and the survey respondent.
But First, Do You Want to Explore or Measure?
When you review the pros and cons, you need to look at both audiences.
At the end of the post, it will be up to you to determine the correct sequence of questions that work best for your survey, including the number of open-ended and closed-ended questions.
As a side note, first, ask yourself if you want to explore or measure your market research study.
Exploring lends more toward open-ended questions and qualitative market research. Measuring lends more toward quantitative market research and close-ended questions.
Learn more about the difference between quantitative and qualitative market research.
Pros of Using Open-Ended Questions for the Survey Writer
Here is a list of pros for open-ended questions for the survey design team.
1. Unbiased responses
Unsure of the exact answer categories to list for your question? This is one of the benefits of free text boxes.
Instead of biasing the respondent through the answer categories offered, an open text box will present you with unbiased feedback.
2. Word clouds
Everyone loves data visualization and graphical displays of data. Collecting open-ended responses provide the survey writer with the flexibility to create a word or image cloud based on responses.
The words used the most are detailed in larger fonts.
Want to learn more about word clouds in market research? Go here.
3. Word counts
Similar to word clouds, open-ended questions provide the analyst with additional flexibility for reporting.
Word counts tally the number of times a specific term was typed in the open-ended comment box (e.g., did you know 67% of customers mentioned the sales rep Dan S. when asked about what they liked best about buying from Company Inc.).
4. Sentiment
Yet even more flexibility in analysis, sentiment addresses the feelings or emotions behind a typed response.
Is the open-ended comment positive or negative? Sentiment analysis helps measure this for the analyst rather than relying on radio button selections with no context.
5. Exhaustive
The MECE principle in market research teaches us being exhaustive is an integral part of survey design.
By using an open-ended comment box, it ensures the question responses will be exhaustive and rich with detail.
A con of using closed-ended questions runs the risk of forgetting to list an important category for selection. If you head down this close-ended path, always include an "other" if warranted.
Cons of Using Open-Ended Questions for the Survey Writer
Here is a list of cons for open-ended questions for the survey design team.
1. Coding
Every analyst's favorite task, right? Probably not. When you ask open-ended questions, you'll be forced to read through all of the responses and categorize them into themes.
These themes are then built into charts and graphs. It requires a lot more work than running a frequency chart on a closed-ended question.
Interested in learning more about coding an online survey? Watch this 1-minute video.
2. Clean-up
Letting the respondent type out a comment will undoubtedly lead to misspellings and grammatical errors. To clean up the data file and report, you'll want to go through and clean these open-ended comments from your respondents. An unfortunate "must-do.”
3. Drop-off
Let's face it, sometimes survey respondents are lazy. Too many open-ended comment boxes, and you'll definitely see some drop-off in your survey.
Survey respondents want it easy, and in many cases, "easy" means select from a choice of radio buttons.
Design your online survey with the user and respondent in-mind. This translates to mobile experiences.
Pros of Using Closed-Ended Questions for the Survey Respondent
Here is a list of pros for closed-ended questions for the survey respondent.
1. Free comment
Frequently respondents like free text boxes because it's free reign for them to type whatever they'd like.
They are not constricted by the categories you've shown previously, and they now have a chance to voice their opinion straight from their heart.
Many times, they like this, and if it's not offered before the end of the survey, they often feel like they could not share the whole story.
2. Engagement
Respondents can only take so many radio buttons, boxes, circles, slider scales, or grid ratings.
Placing an open-ended question in a survey that requires them to think about what they want to write helps promote engagement and interest if not overused.
3. Simplicity
Closed-ended questions make it easy for respondents to answer and for researchers to analyze.
One of the most favored closed-ended questions in market research would be questions relating to the Net Promoter Score (NPS).
For example, a survey question could ask, “How likely are you to recommend this restaurant to your family/friends on a scale from 1 to 10?”
Respondents can provide their honest feedback to this simple but effective question. For the researcher, these questions provide quick numerical responses that measure customer experience and satisfaction.
Cons of Using Closed-Ended Questions for the Survey Respondent
Here is a list of cons for closed-ended questions for the survey respondent.
1. LOI
Open-ended comments take type to type. So several of these added to an online survey will add significant time.
This adds to the length of the interview (LOI). Nowadays, respondents don't have more than 3 to 5 minutes to share feedback.
Surveys need to be quick and to the point. Open-ended questions don't always help with this.
2. Limitations
There are limitations when it comes to answering closed-ended questions in a survey. Closed-ended questions prevent respondents from going in-depth with their answers.
Survey respondents will not be able to elaborate on their responses by explaining their thoughts and feelings in great detail on the topic at hand. There is no room for that in closed-ended questions.
Additionally, there isn’t a way for respondents to ask for clarification. So if a question is confusing to the survey respondent, then they can’t ask for help. Therefore, these questions have limits.
3. Mobile devices
Typing open-ended comments is a lot easier with a keyboard, more difficult with a tablet, and much more difficult with a smartphone.
Considering half of all online surveys are completed via a mobile device, open-ended questions are a large concern because it can become frustrating to type and continually have to go back and correct (for those who care enough to)closed-ended those who don't see Con #2 above for the survey writer.
As a result of this mini-frustration, open-ended questions often drive drop-off on online surveys.
In Conclusion
Strike the right balance of open-ended and closed-ended questions in your survey. The ratio we use internally is about 1 open-ended question for every 10 to 15 questions.
Any more than that and the survey is not designed with the mobile user in mind.
Open-ended comments provide all kinds of flexibility in analysis and reporting, but you must first secure survey respondents and get them to complete your survey before you worry about post-fieldwork options.
Contact Our Market Research Company
Drive Research is a full-service market research company, specializing in both quantitative and qualitative studies. Our team has years of experience writing surveys that benefit both the responent and the client.
To learn more about our services, please contact us through any of the four ways below.
- Message us on our website
- Email us at [email protected]
- Call us at 888-725-DATA
- Text us at 315-303-2040
George Kuhn
George is the Owner & President of Drive Research. He has consulted for hundreds of regional, national, and global organizations over the past 15 years. He is a CX certified VoC professional with a focus on innovation and new product management.
Learn more about George, here.
