Open-ended comments are a powerful feature of the modern employee survey; while the majority of the content for most employee surveys is the quantitative, closed items that employees rate on an agreement scale, there's special value in the open-ended feedback that employees provide. For one, the employees are taking the time to provide thoughtful feedback, which collectively helps managers understand the tone or sentiment of the group, allowing the real voice of employees to shine through. Open-ended feedback also gives employees a chance to offer specific suggestions for improvement or identify hot spots that need to be addressed.
Yet while survey comments provide a goldmine of rich data, too often the use of open-ended comments is sub-optimized. This is in part how open-ended questions are presented on the survey, what type of guidance is given, and then how the project team uses that feedback. When a survey closes, the project team generally has a quick deadline to analyze and present results, and then cascade results through the levels of the organization. It can be difficult to quickly assess and interpret the richness of open-ended text data, especially for large organizations that might have tens of thousands of comments. Thankfully, there are some clear-cut ways to improve your open-ended feedback process:
Use Open-Ended Opportunities Strategically
In the quest for feedback, it’s easy to get excited to think about all the potential ways you get feedback, such as providing an opportunity for people to expand on why they answered each item the way they did. While this might seem like an intriguing idea in the planning stage, it would make the survey too burdensome, and the quality of feedback would likely decline as an employee moved through the survey. A better approach is to think about which items on the survey are the most strategically important to the organization and then offer employees a chance to expand upon their feedback just for those items. Taken further, we often recommend having a pop-up box for additional feedback not just on specific items, but only if a user answered a certain way. For example, if you’re concerned about retention, an item like “I plan to be working at this organization within the next 12 months” might solicit a pop-up only if the user clicks strongly disagree or disagree.
Provide Guidance for Crafting Open-Ended Feedback
OrgVitality has conducted a lot of research into what makes for a useful and actionable comment, and also how to help guide employees as they provide this feedback. We have found that including a specific nudge around writing comments winds up soliciting comments that are five times more actionable than if you didn’t include the nudge. It’s a small tweak that can have big impact; most employees want to be helpful so providing clear guidance makes a big difference.
[Read More: Nudges to Drive Organizational Change]
Conduct Robust Analytics with the Right Tools
Long gone are the days when people had to handle the arduous task of going through thousands, if not hundreds of thousands, of comments by hand. Technology tools like OrgVitality’s proprietary UQ – or Usefulness Quotient – provide consulting-level analytics automatically and at scale. The UQ assigns a 0-100 score to every comment, helping focus leaders on the comments that are the most helpful and actionable. This algorithm scores comments in their native language and has been validated in 17 languages. The tool provides a myriad of ways for leaders to sort through comments, such as filtering by sentiment, topic, usefulness, or keyword. The key for choosing the right technology is making sure that it is customized for your organization. For example, the UQ, which is a form of Machine Learning AI, is customized not just for the overall organization but also manager level or business unit. Other forms of AI are promising to enhance comment analysis even further, with possible applications including comment summaries and more.
[Read More: Generative AI at Work]
There are a lot of other questions we get from clients about various approaches to comment analysis. Some may ask about scrubbing sensitive words, or what to do if a comment identifies an individual. Others are concerned about ensuring confidentiality. Here is where an experienced consultant can help; as with all of our work, we believe that a customized approach is the best way to ensure that your listening strategy is effective, confidential, and impactful. There’s no one-size-fits-all approach to comment analysis. If you’re struggling to implement comments into your listening strategy or with back-end analysis, reach out for a free consultation.
Author
Dr. Victoria Hendrickson is a partner and vice president in the consulting department at OrgVitality. She works to strategically design and administer employee surveys, customer surveys and linkage research. Across these tools, she works to gather data that helps leaders address their organization’s unique strategic challenges and to present findings as an insightful story that guides meaningful change. Victoria comes to OrgVitality with a background in organizational development and leadership development. She conducts applied research on topics of survey comments and organizational ambidexterity and regularly presents at national and international conferences. Victoria received her undergraduate degree in Social Psychology from Saint Mary’s College of California, and her Ph.D. in Organizational Psychology at Alliant International University.
