How do you prevent an incident response that overcorrects and breaks quality?

How to answer this, stage by stage

Nobody is grading whether you know the letters P, I, C, K. They're grading whether you can size a fix to where the evidence actually points, on a question built to tempt you toward the biggest, safest sounding response instead. Seven moves get you there.

One more thing before the walkthrough ends: this isn't a rule that every incident gets the narrowest possible fix. If Corveth and Yarrow only had two people running customer service instead of a hundred thirty, or if the broken flow touched every single email instead of one narrow slice, the math flips, and going broad might genuinely be the cheaper, faster move. Say which situation you're actually in, and you've shown judgment instead of reciting "always start narrow."

Let's learn

The tool is a draft sitting in a text box, waiting for one click. Corveth and Yarrow's customer service floor uses Draftbench to answer email: a rep opens a ticket, Draftbench writes the reply, the rep reads it, fixes what needs fixing, and sends.

Before Draftbench, a rep typed every reply by hand. About nine minutes an email, across returns, order status, price questions, and damage claims. After Draftbench, that dropped to about three minutes, mostly reading and sending. Across sixteen thousand emails a week and a hundred thirty reps, that's the difference between a floor that keeps up and one that doesn't.

Then, ahead of the holiday rush, someone trimmed one prompt template to make drafts shorter and faster to read. The template covered damaged-in-transit claims on items over a hundred fifty dollars, things like small appliances and furniture. The old version told Draftbench to always ask for a photo and to flag anything over that amount for a manager's approval before promising a refund. The new version dropped both lines to save space.

Here's the turn. What happened next wasn't Draftbench getting things wrong in some random, scattered way a rep would catch on sight. It got confidently, cleanly wrong in one narrow place. Every damaged-in-transit draft over a hundred fifty dollars started reading the same reassuring way: refund's on its way, no need to send anything back. No photo request. No approval flag. It read exactly like every other good draft Draftbench had written for fourteen months, so reps under holiday pressure sent it the way they sent everything else.

Nine days in, finance's weekly refund reconciliation, a report nobody was specifically watching for this, flagged the damage-claim category as an outlier. Three hundred forty refunds had gone out on items over a hundred fifty dollars with no photo on file, about two hundred sixty thousand dollars total. When finance hand-checked forty of those claims afterward, seven looked like someone had learned the pattern and was gaming it, filing a second claim within days, describing damage that didn't match what had actually shipped.

At its worst, the honest answer is Corveth and Yarrow got both costs at once: two hundred sixty thousand dollars gone before anyone caught it, and a VP ready to shut Draftbench off for every rep, on every queue, while the whole thing got investigated.

The choice I would take back.

What I would leave alone. The other fifty nine drafting flows Draftbench runs, from order status updates to loyalty questions, weren't touched by this edit and never showed the pattern. Rewriting or re-checking every one of them on a guess would cost weeks for no reason. The forty eight hour audit exists to prove that, not assume it.

The lesson. A rule that lives inside a prompt is a rule a future edit can delete by accident. A rule that lives outside the model, blocking a send instead of asking nicely, survives the next person who trims a paragraph to make drafts read faster.

Same incident, two very different sized costs. One of them shows up on a ticket today. The other one shows up on a spreadsheet two weeks from now.

Now here is the same thing as a story

The short version sits above. Read this one for the Thursday morning phone call that almost decided this the wrong way.

Selene can read a damage-claim email and tell, before she reaches the second sentence, whether the story holds together. Four years running CX quality at Corveth and Yarrow will do that.

Draftbench arrived fourteen months ago, and for most of that time it was the good kind of boring. Reps stopped dreading the damage-claims queue, the messiest one on the floor, because Draftbench pulled the right policy language every time and got most of the reply right on the first pass.

For the first six months, Selene pulled a real sample every week: forty damage claims over a hundred fifty dollars, checked by hand against the photo on file and the manager sign-off. Every week, the sample came back clean. So she trimmed it. Twenty a week, then ten, then, by month eleven, she'd stopped pulling the sample at all. Draftbench's drafts always read like they knew the policy cold, careful, specific, citing the right dollar threshold. There's only so long you keep checking a thing that keeps being right.

Then came the week before the holiday rush. An engineer trimmed the damage-claims template to cut boilerplate and speed up drafting for the busiest queue on the floor. The photo-and-approval line came out with it. Nobody flagged it as risky. In the quick test pass, on a handful of low value claims, it never once produced a wrong answer, because none of those test claims were the kind where the missing rule would have mattered.

For nine days, every damage claim over a hundred fifty dollars got the same reply: refund's on its way, no photo needed, nothing to send back. It read exactly like every other confident, well-written draft Draftbench had ever produced. Reps, already moving fast for the holiday queue, sent it the way they sent everything else. Three hundred forty went out that way. Two hundred sixty thousand dollars, before anyone outside finance's own reconciliation report noticed a thing.

Sherrod Hartsdale, VP of Customer Experience, called Selene at eight that Thursday morning with the number already in his hand. His answer was immediate: turn Draftbench off, for every rep, on every queue, until someone could prove the whole thing was safe again.

Selene understood exactly why he wanted that. Two hundred sixty thousand dollars is a real number, and a kill switch feels like the responsible thing to reach for at eight in the morning with a VP on the phone. But Draftbench ran fifty nine other kinds of replies that week, from order status checks to loyalty questions, and not one of them had shown this pattern. Shutting all of it off wouldn't just stop the bleeding on damage claims. It would hand a hundred thirty reps, on all sixteen thousand emails a week, the nine minute version of every reply again, for a problem that lived in exactly one template.

So here is what Selene took back, and what she told Sherrod on the call. Splitting the refund policy into two places in the prompt, a short version most drafts used and the actual enforcement rule sitting lower down, easy to trim without anyone noticing what they'd cut. That's the decision underneath the incident. Sherrod's instinct wasn't wrong to want safety fast. It was wrong to reach for the size of the fix that matched how bad the number felt, instead of the size of the fix that matched where the damage actually lived.

The replay: patch the one flow, block the send button on any refund promise over a hundred fifty dollars without a photo and a manager tap, and run a forty eight hour audit on the other high-risk flows using the same check. All six come back under fifteen percent. Draftbench keeps running for fifteen thousand nine hundred emails a week that were never wrong, and the fix that actually matters ships in a weekend instead of two weeks.

The part Selene would tell Sherrod, and herself: the kill switch would have made the eight o'clock phone call feel better. It wouldn't have made anyone's Tuesday better for the next two weeks.

The four calls, and the numbers behind each one

This is a yes or no between two responses to the same incident, not a rule for every model mistake Corveth and Yarrow will ever make, so PICK carries the weight here.

And if you want to be sure it really works, try it somewhere else

Quillfeather County runs a 311 line where residents report non-emergency issues: potholes, noise, missed trash pickup, and hazards like gas smells, downed power lines, or blocked fire exits. A drafting tool writes the acknowledgment and closure replies a caseworker sends back.

A similar bug hit the hazard-report flow. A template update meant to shorten routine closures started auto-drafting "resolved, thank you for reporting" replies for hazard reports too, before dispatch had actually confirmed anyone had gone out.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0