Why We Sometimes Show You Less: The Honest Math of Small Samples

Open your feedback results expecting a rich, detailed report and you might find something quieter instead. A few broad themes. No quotes. No tally of how many people said what. If that happens, it is not a glitch, and it is not us being stingy with what people told you. It is a choice we made on purpose, and it is worth explaining why.

When only a few people have answered, detail becomes dangerous. Not to you. To them. So we hold some of it back until enough voices make it safe to share. This post is about that decision, the reasoning behind it, and why we think showing you less is sometimes the most honest thing a feedback tool can do.

The problem with small numbers

Say four people give you feedback and one of them rates a particular area low. You read the comment that goes with it. Even with the name removed, you can probably guess who wrote it. You know who was tough to please, who saw that side of your work, who talks that way. With a group that small, a single pointed comment may as well be signed.

Now imagine that comment is a little raw. The honest kind. The respondent only wrote it because they believed it would reach you blended in with others, not as a note you could trace back to them by Tuesday. If we hand it to you whole, we have broken the exact promise that made them candid in the first place.

That is the bind with small samples. The fewer people who answer, the easier each one is to identify, and the more a frank comment exposes the person brave enough to make it. We dug into how that tracing actually happens in an earlier piece on anonymous versus confidential.

What "enough" means

So we set a line. Below a certain number of responses, we don't show the detailed version of your results. The exact number can shift by campaign, but think of it as a small handful, somewhere around four. It is not a magic figure. It is roughly the point where a comment stops being obviously one person's and starts blending into a group.

Above that line, you get the full picture: themes, the weight behind them, and quotes where the respondent agreed to share them.

Below it, things change in two ways.

Right at the bottom, with only one response or none, we show no synthesis at all. One voice cannot be anonymised. It is just that person, talking. Rather than dress it up as a result, we tell you plainly that more people need to weigh in, and we point you to invite a few more or extend the deadline.

In the middle, with a couple of responses but not quite enough, we switch to a lighter view. You see the broad shape of the feedback and none of the fingerprints. No verbatim lines. No counts. Anything that pins a comment to a project, a date, or a particular relationship gets smoothed out. And we say so, openly, on the screen, so you are never left guessing why the detail looks thin.

Why we tell you it's happening

We could hide the change. Quietly soften things and let you assume you saw everything. Plenty of tools do, and it makes their reports look fuller than they have any right to be.

We would rather be straight with you. When detail is reduced, we put a short note on it that says, in effect: there are few enough people here that individuals could be identified, so we have pulled detail back to protect them. You might find that mildly annoying in the moment. We think it is the more respectful move, in both directions. You learn exactly what you are looking at. The people who spoke keep the privacy we promised them.

The trade you are actually making

Here is the part that feels backwards until you sit with it. A tool that shows you raw quotes from two or three people is not being generous with you. It is spending their safety to make your report feel richer. The detail looks impressive and the cost stays invisible, because the cost lands on someone else.

We are not willing to make that trade on your behalf, and honestly, you shouldn't want us to. The same protection that occasionally frustrates you is the reason the feedback was honest to begin with. People tell the truth when they are sure it cannot be pinned on them. Weaken that, and you don't get more truth. You get a report that looks fuller, built on more careful answers.

There is also a simple fix when the lighter view isn't enough: invite more people. Detail returns as the group grows, not because we flip a switch, but because more voices genuinely make each one harder to single out. The math that forces us to hold back at three is the same math that lets us open up at eight.

What this says about the whole product

A feedback report is easy to pad. Quotes, counts, and authoritative charts all make it feel weightier. The harder discipline is leaving things out when including them would burn someone's trust.

We would rather you occasionally see less and know that every word came from someone who felt safe enough to mean it. That is the version of feedback worth acting on, and it is the only version we are interested in giving you.

The short version

If your results ever look lighter than you expected, it is because not many people had answered yet, and we would rather protect them than impress you. Add a few more respondents and the detail comes back. The restraint isn't a missing feature. It is the feature, doing its job.

---

Trueings protects the people who give you feedback by design: a minimum number of respondents before detail is shown, a lighter view for small groups, names attached only when someone opts in, and a final scrub before anything reaches you. See how it works.