Describe how you would write a blameless postmortem for a model behaviour incident.

A postmortem for a model behavior incident is really this method, run backward from the six week gap to the meeting where the gap got created.

How to answer this, stage by stage

Nobody is grading whether you can say "we did a root cause analysis." They are grading whether you can point a blameless postmortem at a decision instead of a person, for an incident where the thing that broke was confident, not careless. Seven moves get you there.

Let's learn

Here's what happens when a queue built to catch a model's own doubt runs for months and slowly stops needing to. Say we build a voice assistant that takes orders at a drive through speaker and over the phone, and turns what a customer says into an order line the kitchen can read.

Before a tool like this got good, a crew member typed every order by hand while a customer talked over static and a car horn. About two out of every hundred items landed wrong, a swapped side, a missed size, a topping nobody asked for. A reviewer used to pull a real, stratified sample of calls each week, five hundred of them, and check each one against what the store's own inventory actually had that day.

Then a newer model shipped. Overall, it really did get better. Misheard items dropped from about two in a hundred to about one in two hundred. And the number of calls flagged as unsure, the ones a reviewer is supposed to pull and listen to, fell too, from forty a week to three within a month.

Here is the turn. Three flagged calls a week is not the problem. The problem is what a reviewer does with three calls a week that keep coming back clean. She keeps opening the queue for a while, out of habit. Then, without ever deciding to, she stops opening it at all. Nothing broke. A number just got small enough to stop feeling worth ten minutes.

At its worst, that costs far more than the flagged queue ever would have. A model that's sure of itself will confirm an order confidently even when it's wrong, and a wrong but confident order never shows up in a queue built entirely around how unsure the model says it is. Weeks can pass with nobody watching for exactly the case that matters most.

The choice I would take back. When the newer model shipped, all of the reviewer's hours got moved onto the flagged queue, and the old weekly random sample, checked against something outside the model's own opinion of itself, got retired because it had started to feel redundant next to a model that rarely needed it.

What I would leave alone. A single missed topping on a quiet Tuesday order is not worth chasing, and most mistakes really are small and getting rarer. The problem was never ordinary mistakes. It was one specific, confidently wrong answer running unwatched for weeks because the only eyes on the system were the model's own.

The lesson. If the only thing checking a model is a number the model produces about itself, you have not built a check. You have built a mirror.

The team treated the shrinking queue like a dial they could trust a little less over time. It was never a dial. It was a switch, and it had already flipped.

Now here is the same thing as a story

The short version is above. Read on if you want to feel how ordinary the six quiet weeks actually looked from Yola's desk.

Yola Brannigan can tell, inside ten seconds of a recording, whether an order actually went sideways or a customer just changed their mind mid sentence. Two years running the voice quality desk at Corncrake Chicken Co will do that. She built the habit on the old model, back when the flagged queue ran forty calls deep most weeks and she cleared it every Friday afternoon with a coffee and a spreadsheet.

Voxlane 2.1 shipped in March. It was, by every real measure the team tracked, a genuine improvement. Misheard core menu items dropped from about two in a hundred to about one in two hundred. Customers stopped calling back to ask why their order had a burger they never mentioned. Yola's own queue, the calls Voxlane itself flagged as unsure, thinned out with it. Week one after launch, still close to forty. By week four, nine. By week five, three, and it stayed there.

She kept opening it. Fridays, coffee, spreadsheet, same as always, even when there was almost nothing in it. For a while.

What nobody in that March meeting knew was that the same 2.1 release had quietly changed where Voxlane checked whether an item was actually in stock at a given store. The old model pulled live from each store's point of sale system. The new one, tuned for speed during the migration, pulled from a menu cache that synced overnight instead. Small change. Nobody flagged it as risky, because in testing it never once produced a wrong answer.

Two months after launch, a supplier shortage hit the Smoked Honey Drizzle topping at forty six stores in one distribution region. Every one of those stores marked it out of stock in their own system, the same afternoon it ran out. Voxlane never noticed. It kept confirming the topping to callers at those forty six stores, cheerfully, in exactly the same tone it used for every order that went right. "Got it, one Smoked Honey Drizzle added." No hesitation. No confidence score low enough to land anywhere near Yola's queue.

For six weeks, nobody at Corncrake was looking at anything that could have caught it. Yola's queue held steady at three calls a week, all of them unrelated, and somewhere in that stretch she went from opening it most Fridays to not opening it once for a run of six.

The queue's fall was gradual and looked like good news. What she did about it was not gradual at all. It was a switch that flipped once and stayed there.

Roughly forty one thousand orders across those forty six stores included the topping during that window. About nine thousand four hundred of those got caught at the window or in the kitchen and swapped, comped, or refunded on the spot. What made it to a regional ops report was the complaint volume, which tripled at those specific stores, a real number nobody had stopped watching, because complaints were never routed through Voxlane's confidence score in the first place.

Junot Belfrey, the regional operations director for that district, noticed the spike on a Thursday and pulled twenty five random complaint calls to see what was actually going on. Nineteen of the twenty five mentioned the same topping. That's when it reached Yola's desk, and that's the postmortem she had to write.

The old decision, told as a memory of a meeting. When 2.1 shipped, someone on the team pointed out that the weekly five hundred call random sample almost never turned up anything the flagged queue hadn't already caught, and asked whether it was still worth the hours. The room agreed it wasn't. Redirect the time to the queue, where it was clearly doing more work per hour. Nobody in that room was wrong about what the data showed them that week.

The postmortem Yola actually wrote did not put her own name anywhere near the cause line. It put the retired sample there instead, and next to it, the specific technical detail: the per store inventory check and the model's own confidence score live in two completely different systems inside Voxlane, one wired to the point of sale feed, one to the language model's self rated certainty, and nothing had ever connected them.

The replay, run the same six weeks forward with the fix already in place: a new, smaller, stratified sample, a hundred and fifty calls a week, spread so every one of the two hundred and sixty stores gets checked against its own real, same day inventory on a fixed rotation, independent of anything Voxlane says about how sure it is. In week one of that redesigned sample, roughly three of the hundred and fifty calls already land at one of the forty six affected stores, confirming the topping that store had marked out of stock that same morning. Caught in about four days. Not six weeks, not forty one thousand orders, not a hundred and eighty seven thousand dollars in refunds and remade meals.

The thing Yola would tell her past self, back in that March meeting: a sample that almost never finds anything is not the same claim as a sample that isn't needed. It just means the thing it exists to catch hasn't happened yet.

What a postmortem for a model incident actually has to find, letter by letter

This is a perturbation question dressed as a process question. Something changed, the model got genuinely better, and a person's behavior around it flipped as a result, so FLIPS fits, run in reverse from the incident back to the meeting that caused it.

And if you want to be sure it really works, try it somewhere else

Thistlecove Mutual runs Firstword, a phone based intake bot that takes the first report of a car accident, before an adjuster ever gets involved.

F. Wrenna Sutcliffe, the first notice of loss coordinator, four years on the desk, able to tell a clean rear end claim from a disputed multi car pileup in the first fifteen seconds of how someone describes it.
L. Callers used to describe a crash however it actually came out, messy and real, and Firstword parsed it into the right claim fields on the first pass most of the time. Wrenna stopped coaching callers on how to talk to it, because it stopped being necessary.
I. A different flip than Voxlane's. This one is an input flip. After Firstword locked in the wrong at fault driver on a run of multi vehicle calls, and only ever followed up with a yes or no confirmation instead of an open question, Wrenna started coaching callers to talk like a form. "Vehicle one struck vehicle two, low speed, one lane." Once a caller learns to flatten their own story like that, they never go back to describing it naturally, on this call or the next one they ever make.
P. Firstword's team chose yes or no confirmations over open repair prompts, because in early testing, mostly single vehicle claims, confirmations scored faster and cleaner. Reasonable, for the claims they tested it on.
S. Swap one line: whenever the parse confidence drops mid call, ask an open question instead of a yes or no one. "Sorry, can you say what happened again, in your own words?" Callers keep describing naturally, and the share of complex claims with the right fault field on the first pass climbs from about sixty one percent to about ninety percent.

Swap the trigger and it still runs.
Speed: an interviewer caps the answer at ninety seconds. Skip straight to naming the decision that removed the one independent check, and what replaces it.
Cost: the review team's headcount gets cut in half this quarter. Don't cut the independent sample to zero to save the hours. Shrink it and stratify harder instead, fewer stores checked each week but each one checked fully, on a fixed rotation. That's a stated tradeoff, not a silent one.
The model got better, for real, everywhere: suppose the 2.1 migration hadn't broken the per store feed at all, and Voxlane really was better across the board, including on rare items. That still wouldn't make the independent sample worth cutting. It's the only way anyone would know that for certain, instead of assuming it from a queue that quietly went quiet.

Where people run it wrong.
They point the postmortem at the person who stopped checking, instead of at the decision that removed the one check that didn't depend on the model's own opinion of itself.
They "fix" the incident by retraining the model on the one gap that got found, and close the postmortem there, without asking what else the model might one day be confidently wrong about that nothing is currently watching for.
They read a falling flagged queue as proof the model is getting better everywhere, without ever asking whether it might be drifting toward confident wrongness on some other slice of orders instead.

How to use it live. Say the reframe before any story: "the point of a blameless postmortem for a model behavior incident is that the thing being audited rated its own homework and passed itself, so the fix isn't a person, it's the decision that let a self graded system grade the one exam nobody else was watching." That buys you the room to name the actual decision, instead of reciting "communicate clearly and hold a retro."

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0