How do you decide which actions require confirmation and which do not?

Ferrant Metal Works runs three auto-parts stamping lines. A maintenance agent watches vibration and heat sensors on each press, and can act on what it sees: reorder wear parts, pause a press, or, if it decides things are serious enough, order a rush replacement and shut a whole line down. Dabir Osei has supervised maintenance there for eleven years, and carries a radio that pages him whenever the agent needs a human.

How to answer this, stage by stage

Nobody's grading whether you can list every action an agent might take. They're grading whether you can commit to one rule and defend the asymmetry behind it.

Let's learn

What decides whether an agent asks first or just acts: is the mistake cheap to undo, and is it cheap in dollars. Get those two questions right and the rest of the policy writes itself.

Ferrant Metal Works' maintenance agent watches sensors on three stamping presses and can reorder parts, pause a press, or push for a rush order and a full shutdown, without Dabir's team touching it first.

Before the agent, every one of the roughly 25 daily sensor alerts went through a person: Dabir's team reviewed each one, about three minutes apiece, and decided what to do. Nothing happened without a human saying so.

The agent now auto-handles about 22 of those 25 a day on its own: lubrication reminders, small parts reorders under 200 dollars, brief calibration pauses. All cheap, all reversible. For months, that was the whole story, and it was fine.

Then came a Saturday at 2:14 a.m. A loose sensor mount on Line 2 threw a false vibration spike, not real bearing wear. Under a single blanket approval threshold that treated every order type the same, the agent auto-approved an $18,400 rush-freight replacement motor and triggered a 14-hour full-line shutdown, with nobody woken up to check first.

What I would leave alone: the 22 cheap, reversible actions a day. Adding a confirmation step there would just rebuild the manual review the agent was supposed to remove.

The lesson: a single dollar number feels like a safety rule, but it only works if every action behind it actually costs the same to get wrong. Ours didn't.

Now here is the same thing as a story

The short version above is what you'd say defending this rule to Ferrant's plant leadership. Read this one for how the gap actually got found.

The control room at Ferrant Metal Works goes quiet around midnight on a Saturday. Two people on shift, a bank of monitors, and a radio that pages Dabir Osei if anything real comes up.

Before the agent, every one of the 25 daily alerts passed through a person before anything happened.

For six months after launch, the agent handled the routine stuff cleanly: lubrication reminders, small reorders, a calibration pause here and there. Dabir stopped reviewing the daily action log around month three. It always said the same thing: handled, handled, handled.

Nothing built up slowly here. Six quiet months, then one number nobody had split in two.

At 2:14 a.m. that Saturday, a loose sensor mount on Line 2's press threw a vibration reading that looked exactly like early bearing failure. The agent, running under a single company-wide auto-approve limit of 20,000 dollars that never distinguished routine bulk spend from a one-off emergency spend, cleared an $18,400 rush-freight motor order and started a full shutdown, both before the number crossed its one threshold.

One of these costs a text message. The other costs a day's production. They were never the same kind of mistake.

Dabir found out at 8 a.m. Monday, not from a page, but from a production report showing Line 2 dark since Saturday. The sensor mount, once checked, was loose, not failing. The motor had been fine.

Once plotted, the gap was obvious: routine actions cluster in one corner, and the two dangerous ones sit far apart from them, in the other.

Dabir's team hadn't lost trust in the routine 22-a-day lane; that part kept working exactly as designed. What broke was the assumption that one dollar line could safely gate every action type at once. A rush order at 3 a.m. is not the same kind of decision as a scheduled bulk parts reorder, even when the two numbers happen to sit near each other.

The fix wasn't a smarter model. It was splitting one number into a rule that actually asks two different questions.

Replayed with the split rule in place: the same false spike on Line 2 still reaches the agent, but a rush order above the new, separate emergency-cost line now waits for Dabir's confirmation. His radio pages him at 2:14 a.m. He checks the feed, sees the numbers don't match a real failure pattern, and holds the order. Six minutes of his night, instead of $18,400 and 14 hours of a dark line.

Everything on this short list shares one trait: expensive and hard to walk back if the agent guessed wrong.

We built one blanket number because separate thresholds per action type felt like extra configuration nobody would thank us for. It took a quiet Saturday morning, and a production report nobody wanted to write, to see that "one number" and "one kind of risk" were never the same thing.

PICK, the confirmation rule in one screenNot a vibe about how careful to be. PICK is what forces the rule into something you can actually defend.

The recap, one line per letter: position is confirm-before-costly, impact is six minutes against sixty thousand dollars, cost asymmetry is optimizing against the silent wrong approval, and kill criteria is watching whether confirmations start taking too long to answer.

And if you want to be sure it really works, try it somewhere elseSame rule, an insurance claims desk instead of a stamping line. No factory in sight, and the asymmetry still holds.

Colm Meade leads a claims-adjustment team where an agent reviews incoming auto-claims: it can request another document, auto-approve a small routine payout, deny a claim outright, or flag one for fraud review. Mapped onto PICK: the position is the same shape, confirm before costly or hard to undo, auto-proceed otherwise. A routine payout under 300 dollars with clean documentation auto-approves and gets logged. Denying a claim outright, which starts a legal appeal clock the moment it's sent, always waits for an adjuster's confirmation, since an unsendable denial is easy to fix but a sent one is not. The impact: over-confirming costs an adjuster a few minutes reviewing something that was fine. Under-confirming on a wrongful denial can cost the company a compliance complaint and weeks of appeal handling. The cost asymmetry favors confirming denials every time, even though most of them turn out fine on review.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "confirm on cost and reversibility, not confidence," and stop.
Cost: there's no budget this quarter for a fancy approval workflow. Route the confirmation through whatever channel already reaches an on-call person fastest, a page, a text, a phone call.
The model gets better, for real: if the agent's accuracy improves overall, that's still not a reason to drop confirmation on the costly, hard-to-undo actions. A better average model can still guess wrong on the one case that costs sixty thousand dollars.

Where people run it wrong.
They set one dollar threshold for every action type, the exact mistake that cost Ferrant $18,400.
They confirm everything the model flags as "uncertain," which trains people to stop reading confirmation requests at all.
They never revisit the rule once volumes or action types shift, so a threshold that was safe at launch quietly stops being safe.

How to use it live. When someone asks how you'd decide what needs confirmation, answer with the two questions before naming a single example: is it cheap to undo, and is it cheap in dollars. Everything else is just sorting real actions into those two boxes.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0