Explain when a human checkpoint is theatre rather than control.

Cinderford Home Pharmacy fills prescriptions by mail and runs an AI model that flags likely drug interactions before a pharmacist confirms and dispenses. Naledi Dlamini has verified those flags for four years, and used to open the evidence behind every single one before she clicked confirm.

How to answer this, stage by stage

Nobody is grading whether you can define "rubber stamping." They're grading whether you can name the exact measurement that tells theatre and control apart.

Let's learn

Say Cinderford Home Pharmacy fills prescriptions by mail. An AI model reads every new order against the patient's medication history and flags anything that looks like a risky interaction, before a pharmacist confirms and it gets dispensed.

Before the model, a pharmacist checked every single new prescription against the patient's chart by hand. Thorough, and slow, several minutes each.

Now the model flags about sixty interactions a week out of thousands of orders, and a pharmacist reviews each flag before confirming.

The fade looks gradual from the inside. From the outside, it has exactly two settings.

Here's the turn: the flags themselves were never the problem. The model stayed accurate. What changed was what a pharmacist did with a flag once opening the evidence behind it stopped feeling necessary.

At its worst, a genuinely rare interaction, the kind the model itself flags with less confidence, gets the same reflexive click as the sixty routine ones that week. The checkpoint is still there. It just isn't checking anything anymore.

What I would leave alone: refill confirmations for medications with no interaction history at all don't need any of this. There's nothing risky hiding behind those, and a fast, light checkpoint there is honestly fine.

The lesson: a checkpoint that never changes its own pace, no matter what's behind the flag, was never really a checkpoint. It was a click wearing one.

Now here is the same thing as a story

The short version above is what you'd say to a pharmacy board reviewing the interaction workflow. Read this one for the two seconds that almost cost someone.

Naledi Dlamini has verified drug interaction flags at Cinderford for four years. She can picture the mechanism behind most common interactions without looking anything up, from memory alone.

No single month looks alarming. The whole line does.

When the interaction model launched, she opened the evidence panel behind every flag, read the mechanism, and confirmed or escalated. It always checked out. Over about a year, without any one moment marking the change, she opened it less. Then rarely. Then, without quite noticing when it happened, not at all.

A pharmacist new to the mail-order side, shadowing her one afternoon, watched her clear a stretch of flags in seconds each and said, half-curious, "You don't even open those anymore, do you?" Naledi laughed it off. She wasn't wrong, though, and the comment sat with her longer than she expected.

Naledi's checking never had a dial. It only ever had two positions.

That same week, a flag came through for a returning patient starting a new antidepressant while already on a painkiller, a pairing the model scored as medium confidence, lower than its usual all-clear. Her hand was already moving toward confirm, the same two seconds as the sixty routine flags that week, when the comment from days earlier snagged something, and she opened the evidence instead.

The panel showed what the model's training data hadn't: the patient's intake notes mentioned a supplement, the same one that pushes that exact pairing into genuinely dangerous territory, information sitting in a free-text field the model wasn't weighing. She held the fill and called the prescriber.

With the redesigned screen, the evidence panel has to actually open, a real two-second minimum view, before the confirm button activates at all. Run the same week forward with that in place: the medium-confidence flag can't be waved through in a reflex, because the reflex itself no longer has anywhere to land.

I let the click get faster because faster looked like the system working. It took a new colleague's offhand question, and a supplement two lines down in a chart, to see that a fast click and a real check had quietly stopped being the same thing.

FLIPS, the over-trust version, in one screenNot the usual "model got worse" story. This is what happens when it gets better.

The same five steps, whichever direction the model's accuracy is moving.

The recap, one line per letter: find is Naledi at her screen, locate is the evidence-panel habit thinning to nothing, identify is the over-trust flip with no stable middle, pinpoint is the one-click merge that made sense before precision improved, and show is a re-split checkpoint that catches the case the old one would have waved through.

And if you want to be sure it really works, try it somewhere elseA different flip family entirely, a legal translation firm instead of a pharmacy.

Meridian Legal Translations uses an AI model to draft first-pass translations of contracts, which a human translator then reviews before delivery. Emil Rutkowski has translated legal documents there for eight years.

Mapped to a different family, the concealment flip: when the AI drafts started, Emil marked every delivered document "AI-assisted, reviewed by Emil," proud of the speed. A few embarrassing early drafts, an AI mistranslation of a liability clause that a client's own lawyer caught, made that label feel like a confession instead of a badge. He quietly stopped marking anything, delivering AI-assisted drafts as fully his own work. Nobody flagged errors to the team anymore, since nobody was owning up to using the tool in the first place, and the firm's own error-tracking data went dark exactly when it needed to see the most.

A completely different flip family. The same theatre problem shows up anyway, one step further upstream.

The upper right corner is where a fast, uniform click is the most dangerous kind of theatre.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "a checkpoint is theatre when review time stops varying with real risk, whatever the reason the person stopped truly looking," and stop.
Cost: there's no budget to redesign the confirm flow this quarter. Say so honestly, and start by simply logging open-versus-not-opened on the evidence panel, since that number alone will show whether the checkpoint is still real.
The model gets better, for real: if accuracy keeps improving, the temptation to skip the check only grows, which is exactly why this failure mode gets more likely, not less, as a model gets better.

Where people run it wrong.
They watch the confirm rate and call the checkpoint healthy, when a rubber-stamped confirm looks identical to a genuine one on that number alone.
They assume a checkpoint that catches nothing this month is a checkpoint that's working perfectly, instead of asking whether it's still capable of catching anything.
They treat "the click still happens" as proof of oversight, when the click was never the thing doing the checking.

How to use it live. When someone asks about theatre versus control, ask one question first: does this checkpoint take longer on the case that deserves it? If the honest answer is no, you've found the theatre, before you even need the story.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0