How do you close the loop so users see that their feedback mattered?

Interviewer's question: "How do you close the loop so users see that their feedback mattered?" Alder Hollow township runs CivicLine, a chatbot that answers questions about trash pickup, permits, and local services. Greta Solheim has used it every week since it launched.

How to answer this, stage by stage

Nobody is grading whether you sound caring. They're grading whether you can name the exact record that turns a flag into a kept promise.

Let's learn

Picture reporting the same mistake three separate times and hearing nothing back, not even once.

CivicLine is Alder Hollow township's chatbot. Residents ask it when trash pickup is, whether a permit is needed for a fence, what the library's summer hours are, and it answers in a couple of sentences, day or night.

Early on, residents flagged wrong answers often, about forty a month, using a thumbs-down button with a short comment box. The team read every one.

Then, over a few months with no single bad event marking it, flags quietly dropped from forty a month to nine. Nobody complained about the drop. Nobody complained at all. That was the actual warning sign, and almost nobody read it as one.

Here's the turn: the falling flag count was never the real problem. It looked like people had simply stopped noticing mistakes. What actually happened is they kept noticing them, and kept deciding, one flag at a time, that saying so didn't do anything.

The fading happened slowly. The stopping happened all at once, and quietly.

At its worst: a wrong answer sits uncorrected for months because nobody flags it anymore, a resident acts on it, misses trash pickup or shows up to a closed office, and the township never even learns the answer was wrong in the first place, because the one channel built to catch it went quiet first.

What I would leave alone: the plain thumbs-up button stays exactly as light as it is. It never needed a reply, since agreeing an answer was helpful isn't a report that requires closing any loop.

The lesson: a feedback button that never reports back isn't neutral. It slowly teaches people that using it was pointless, and that lesson sticks long after the actual bug gets fixed by someone else, some other way.

Now here is the same thing as a story

The short version above is what you'd say defending this design to Alder Hollow's town manager. Read this one for how the silence actually built up.

For eight months, the best small habit in Greta Solheim's week was checking CivicLine on Sunday night before recycling day. It was usually right, and on the rare week it wasn't, she'd flag it and move on, never thinking twice.

There was no dramatic failure. CivicLine kept answering most questions correctly, same as always.

Four small beats. Nothing about any single one of them looks like a warning.

What actually happened was smaller and slower than a failure. CivicLine told her, three separate times over two months, that compost pickup had moved to Thursdays. It hadn't. She flagged it the first time, mentioned the exact wrong day in her comment. Nothing changed. She flagged it again a few weeks later, same wrong day, same silence. The third time, she typed the comment, hit submit, and just stood there for a second, phone in hand, before putting it back in her pocket.

Nine weeks. No single one of them looks like the moment anything broke.

She never complained about it to anyone at the township. She just quietly stopped flagging things. Not out of anger. It had simply stopped feeling like it did anything.

Here's the decision I'd take back: when the flag button shipped, we never built a record connecting a specific flag to whatever fix eventually landed, if one ever did. That felt like a reasonable corner to cut at launch, since the team was small and fixes happened fast enough that it didn't seem to matter yet. It stopped being reasonable the day fixes started taking weeks and nobody could tell Greta hers had.

Replayed with that link in place: Greta flags the wrong compost day. The fix lands four days later, a real correction to the underlying schedule data. The system checks which open flags pointed at that exact fact, and sends her a short text: "You were right, compost pickup is Wednesdays, not Thursdays. Thanks for flagging it." She reads it standing in her kitchen, and the next time CivicLine gets something wrong, she flags it the same way she always used to, without a second thought about whether it's worth the ten seconds.

The old design assumed a fix landing anywhere was enough. The new one makes sure the fix finds its way back to the specific person who asked for it.

I signed off on skipping that link because it felt like a small piece of plumbing nobody would notice missing. It took watching a genuinely careful, patient resident go quiet over nine weeks to see that the plumbing was the entire point.

The five steps, if you want to remember itNot a customer-service script. FLIPS is what tells you silence, not anger, is what actually breaks a feedback loop.

Nobody designed a dial for how often she'd flag something. There was never a dial there to begin with.

Five small facts. Together they're the whole answer.

The recap, one line per letter: find is Greta, a careful weekly user, locate is the habit of flagging without hesitation, identify is the flip from flagging to total silence, pinpoint is the missing link between a flag and its fix, and show is the four-day notice that brought the habit back.

And if you want to be sure it really works, try it somewhere elseSame five letters, a veterinary clinic instead of a township chatbot, and a different flip family entirely: this time the model gets a shiny new metric, and that's what breaks the loop.

DoseNudge is a tool vet clinics use to suggest medication dosages, which staff can flag if a suggestion looks off. Renata Cho is a vet tech at a clinic using it.

For months, Renata double-checked every flagged dosage suggestion herself before trusting it, the same way any careful tech would. Then DoseNudge added a badge to its dashboard: "94% of flags resolved within a week." It was true, in aggregate, and it was meant to build confidence.

Mapped onto FLIPS: find is Renata, a vet tech who's always double-checked flagged suggestions herself. Locate is that habit of checking, done every single time, no exceptions. Identify is the flip, and this time it runs the other way: seeing the 94% badge, staff stopped double-checking specific flagged suggestions themselves, assuming the badge meant this one had already been handled. That's an over-trust flip, not an abandonment one, the opposite direction, triggered by something that looked like good news. Pinpoint is the decision that made it possible: the badge showed only an aggregate resolution rate, with no way to see whether this specific flagged item had actually been resolved yet. Show is the fix: replace the aggregate badge with a per-item status next to each flagged suggestion, so trusting it again requires seeing that this one, specifically, was checked.

One good-looking number, shown the wrong way, undid months of careful habit almost overnight.

Swap the trigger and it still runs.
Speed: an interviewer caps you at thirty seconds. Say "link every flag to its fix, and notify the specific person the moment it lands, or the loop never really closes," and stop.
Cost: if building a full notification system is too expensive this quarter, ship a simple weekly digest of "flags you sent that got fixed" first, since even a delayed, batched notice beats permanent silence.
The model gets better, for real: if CivicLine's underlying accuracy improves so much that flags become rare, that's still not a reason to drop the notification habit, the rare flag that does come in matters even more when it's one of the only ones left.

Where people run it wrong.
They build a feedback button and treat "we read every submission" as if it were the same thing as closing the loop.
They send a general product-update newsletter and assume it counts as a reply to specific complaints buried somewhere inside it.
They measure flag volume as a health metric without noticing that a falling number can mean people gave up, not that things got better.

How to use it live. When someone asks how to close a feedback loop, ask yourself one question first: can I trace a straight line from this specific flag to a specific fix to a specific notice back to the person. If any link in that chain is missing, that's the actual gap to fix, not a vaguer promise to "listen better."

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0