What Your Engagement Survey Isn't Telling You (But the Open Comments Are)

I asked a room full of employees at a startup last spring whether they felt like they belonged at their company. About 60 percent said "sometimes." That word has been sitting with me ever since.

Not "no" — which would have been alarming and easy to act on. Not "yes" — which would have felt like a success. Sometimes. Which means something is working, something isn't, and we don't quite know which is which yet.

That word came from an open-ended comment on an engagement survey. The quantitative scores were fine — nothing that would have triggered a conversation with the leadership team. The 5-point scale items looked stable. But in the comments, employees were telling a different, more complicated story. And almost nobody was reading them.

The Gap Between Scores and Reality

Engagement surveys are among the most widely used HR tools in organizations today — and among the most misread. The issue isn't the tool itself. Quantitative engagement scores give you something genuinely useful: trend data, benchmarks, the ability to track change over time across a large population. That's valuable and I'm not arguing against it.

The problem is what gets measured versus what matters. Most engagement survey scales ask employees to rate agreement with statements like "I have what I need to do my job well" or "My manager gives me feedback that helps me improve." These questions were designed to capture broad, org-wide patterns. They were never designed to tell you why an individual employee in Accounting feels unseen, or why the newest members of your Customer Success team are already mentally halfway out the door.

Open-ended questions — "What would make this a better place to work?" or "Is there anything you'd like us to know?" — get at that layer. But in my experience, they're also the questions most likely to be skipped in analysis. The comments are messier. They take longer to read. They resist the clean dashboard summary that makes it easy to present findings to leadership in a 15-minute slide deck. So they get skimmed, or summarized at a high level ("employees want more flexibility"), or left for "when we have time."

What to Actually Look for in Open-Ended Responses

Over several years of building and analyzing engagement surveys at startups and mid-size companies, I've found that open-ended comments are most valuable when you're reading for patterns of feeling, not just patterns of content.

The content is what employees say they want — more flexibility, better communication, clearer career paths. That matters. But the feeling underneath the content tells you whether the organization's relationship with its employees is fundamentally healthy or not.

A comment like "I wish we had more opportunities to grow here" reads very differently from "I keep asking about growth and nothing ever happens." The surface content is similar. The relationship signal is completely different. One is a suggestion. The other is an early exit interview.

A few specific patterns worth flagging when you read through comments:

Language about being heard versus being asked. Employees who feel heard say things like "I appreciate that leadership actually responded to last year's survey." Employees who feel asked but not heard say things like "I'll be curious to see what happens this time." The difference is subtle. It matters enormously for trust.

Inconsistency between individual scores and individual comments. If an employee rates "I feel valued at this company" as a 4 out of 5 — but their open-ended comment describes feeling passed over for a project or left out of a key meeting — pay attention to the comment. People often score more generously than they feel. The comment is closer to the truth.

Silence from specific groups. If your demographic breakdown shows lower comment submission rates from one group — by tenure, by department, by identity — that is itself a signal. Psychological safety research consistently shows that employees who don't feel safe are less likely to speak up, even in anonymous surveys. Low comment rates from specific groups may not mean "they have nothing to say." It may mean "they don't believe it's safe to say it."

Making Time for This Without Losing Your Mind

I understand that reading every open-ended comment on a 500-person survey sounds like a full week of work. It doesn't have to be. A few things that help:

First, use thematic coding — even rough, manual coding — to group comments by topic. You don't need fancy text analysis software. A shared spreadsheet with five or six theme categories and someone spending two focused hours on it can give you a much cleaner picture than skimming and hoping.

Second, don't analyze in isolation. Bring two or three people from different backgrounds into the reading process. I've consistently found that I notice different things in comments than a colleague who came up through a different function or has a different lived experience than I do. That diversity of readers makes the analysis richer and less prone to confirmation bias.

Third, share a summary of what you found in the comments — not just the scores — with leadership. Not a data dump. A brief, honest narrative: "Here are three things the scores didn't tell us that the comments did." If that becomes a regular part of your engagement reporting, it changes the conversation about what the survey is actually measuring.

Engagement scores will tell you whether something is moving. The open-ended comments will tell you why. And in my experience, the "why" is almost always more interesting — and more useful — than the number.

What do you do with the open-ended responses in your surveys? I'd really like to know — I think there's a lot of practice wisdom on this that doesn't get shared enough.

Questions? Reach out anytime.

support@workplayengage.store

WorkPlayEngage is a product of Dieza Publishing. © 2026 WorkPlayEngage. All rights reserved.

Privacy & Security Note: WorkPlayEngage templates are native Google Sheets. We do not host your data, and we never have access to your employee names or internal metrics. Your engagement remains 100% private within your organization’s Google Drive.

Play:
Baby Pool
NFL Survivor Pool (Coming Soon)
Super Bowl Squares (Coming Soon)
Office Bingo (Coming Soon)

Learn:
How It Works
Shop
Baby Pool
Our Story

Blog:
Employee Engagement
HR Operations
Office Culture
Games & Activities
Whitepapers & Guides

Help

WorkPlayEngage:
Our Story
Contact Us
Blog
Privacy Policy