Most employee listening strategies include a mix of open-ended questions. Yet, how do organizations choose - and word - their questions to solicit the most helpful feedback? OrgVitality Partner Amanda Dundas sat down with her colleague Dr. Victoria Hendrickson, author of Needle in the Haystack: Finding and Acting Upon the Most Useful Comments, about crafting the most impactful open-ended employee survey questions.
Much of your research your research has focused on identifying the most useful and actionable comments in the employee listening space. What have you learned about crafting comments in the process?
One thing that stuck out to me in my research is that the first comment, no matter what it is, tends to get answered more negatively than the following comments. Employees are moving through a survey, answering many items on various dimensions, and somehow when they get to that first open-ended question it’s their opportunity to really share a lot of the frustrations they may have. Sometimes survey questions remind them of a particular event they want to discuss, or sometimes they have been thinking about an issue for a long time and waiting for a survey to vent and share that frustration. So, they take the first chance they get.
Knowing that, does it change the order of your open-ended comments on the survey?
Absolutely. I advise our clients to really lean into it. Typically, I recommend structuring the first open-end based on some kind of suggestion for improvement. You will likely solicit a lot of really valuable, actionable feedback. Otherwise, if you ask for strengths or something really focused, you will just get comments that don’t relate to the question asked.
Do you always recommend including open-ended comments?
Unless there is a really strong reason not to, I do recommend them. People want to have their voices heard; often they want the chance to explain why they are answering survey items in a particular way. Also, while the quantitative items let you measure trend over time, qualitative comments often provide specific ideas, insights or suggestions you won’t get otherwise.
That being said, I don’t recommend having an open-ended comment for every single item on a survey. It’s unwieldly for both employees and leadership. A better way is to focus on a few items that are critical to the organization’s strategic priorities and have a pop-up box appear if an employee answers negatively. So, for example, if retention is an issue, you might want an open-ended comment if someone chooses “disagree” or “strongly disagree” to the item, “I see myself still working here after 12 months.” This type of strategic deployment helps you focus on critical issues and ultimately take action that will have impact.
[Related: Tips for Soliciting and Analyzing Open-Ending Comments]
So, what are the factors when choosing which open-ended comments to include?
As with choosing survey items, we start with the organization’s strategic priorities. But this is an opportunity to also look at the areas where a more detailed or nuanced answer can be incredibly helpful. As I mentioned before, comments that actively solicit suggestions such as “What’s one thing that we could improve upon,” are instrumental in bringing the voice of the individual employee to leadership, especially when the tools are in place to adequately review the comments. For example, at OrgVitality we offer both the UQ – a machine learning algorithm that scores each comment for how useful or actionable it is – as well as an AI Comments Summary tool that gives an overview of what the comments are saying. Taken together, these tools focus leadership on the comments that matter and help them use the feedback to meaningfully drive action.
Want help with your employee listening program, including crafting meaningful open-ended comments? Schedule a free consultation to get started.