How should the interface change when the model's confidence is very low?

Meridian Regional Hospital's radiology department uses ScanDraft, an AI tool that drafts a first-pass report from a scan before a resident and attending review it. Dr. Esperanza Cuartas is the senior attending. Bram Vandersteen is the resident she supervises most closely.

How to answer this, stage by stage

The interviewer isn't testing whether you know confidence scores exist. They're testing whether you know a number alone doesn't change anyone's behavior.

Six months of good, then the snap

Every morning, Dr. Esperanza Cuartas walked the reading room floor before the residents arrived, checking which studies had come in overnight. For years, that meant she'd personally read every image herself, about 18 minutes a study, roughly 20 studies a day.

Five steps, and only one of them is genuinely hard to find.

ScanDraft drafts a preliminary report in under 90 seconds. Once Bram and the other residents started reviewing and editing those drafts instead of reading every image cold, Dr. Cuartas's own co-review time per study fell to about 5 minutes.

Here's the turn: the co-review time Dr. Cuartas got back was never the real story. The real problem was a draft that read exactly as confident on an ambiguous mass as it did on a routine fracture, so nothing told a resident which one actually needed a slower read.

A dial has room for nuance. A flip only has room for a person to notice, or not.

At its worst, a flat report design doesn't just cost an extra hour of careful reading. It lets an ambiguous, genuinely uncertain finding slide through as if it were routine, delaying a diagnosis by weeks until something else forces a second look.

What I would leave alone: a clean, high-confidence fracture read doesn't need this same visible hedging. Slowing every routine read down to match the ambiguous ones would waste the exact time the tool was built to save.

The lesson: a confidence number sitting quietly beside confident-sounding prose doesn't protect anyone. The prose has to change too, or the number is just decoration nobody has time to read.

Now here is the same thing as a story

The short version above is what you'd say defending this redesign to Meridian's chief of radiology. Read this one for how the actual near miss unfolded.

Bram had been under Dr. Cuartas's supervision for eight months, and the arrangement worked the way good training is supposed to: he drafted, she reviewed closely at first, then more lightly as his judgment proved itself case after case. ScanDraft made both of their jobs faster without changing that trust.

Six months in, Bram was reviewing ScanDraft's drafts quickly and confidently, and Dr. Cuartas's spot-checks kept coming back clean. Nothing about the reports themselves ever looked different from one case to the next, so neither of them had reason to slow down on any particular one.

Six good months hid a habit that had already changed. Month eight is where the flat design ran out of luck.

The trigger was a single case: an ambiguous mass on a chest scan, drafted by ScanDraft at genuinely low confidence, worded in the same clean, declarative sentences as every clear fracture read that week. Bram signed off in the time he'd normally spend, since nothing on the page suggested this one deserved longer.

Dr. Cuartas caught it four days later, on an unrelated follow-up read of the same patient, not because any part of the system had flagged it. When she pulled ScanDraft's internal confidence log, the mass finding had scored well below the model's usual range, a fact that never surfaced anywhere Bram could see it.

Four missing signals. Any one of them would have told Bram to look twice.

Here's the flip: Dr. Cuartas didn't just tighten the review process. She pulled every delegation back, insisting on personally reading every scan herself again before any resident touched it, the same as before ScanDraft existed. Two people were now doing the work of one, and Bram's training stalled exactly where it had been going well.

The model's miss rate barely moved. What changed is whether anyone could see which three needed a second look.

The real fix wasn't more review. Meridian rebuilt ScanDraft's report card: any finding below a set confidence threshold renders in hedged language, "findings are equivocal, recommend independent read," inside a visibly bordered section, and the report can't finalize until a named second reader, not just Bram, explicitly signs that specific section.

The five steps, if you want to remember itNot a general lesson about trusting AI less. FLIPS is what finds the exact moment delegation stopped being safe.

Only one case type sits in the corner that actually needs a different screen.

The recap, one line per letter: find the person is Dr. Cuartas and Bram's supervision relationship, locate the habit is her trust in his review replacing her own independent read, identify the flip is delegation reversing into full takeover, pinpoint the old decision is the uniform confident-sounding report design, and show the replay is the hedged, bordered, second-reader flag that lets delegation survive the exact case that used to end it.

And if you want to be sure it really works, try it somewhere elseSame five steps, a contract-review team instead of a reading room. A different flip family breaks the second story.

Vexley & Corbind, a mid-size law firm, uses ClauseKey, an AI tool that drafts plain-language summaries of contract clauses for paralegals to review before an attorney signs off. Odele Frantzen is a paralegal there. Mapped onto FLIPS, but landing on a different family than Dr. Cuartas's story: find the person is Odele, locate the habit is feeding ClauseKey full, messy real contracts the way she always had.

The flip here is a pre-editing flip, not a delegation flip. After a few uncomfortable misses on long, tangled indemnification clauses, ClauseKey's uniformly confident-sounding summaries gave Odele no way to tell which clause needed a closer read. Rather than take the whole review back the way Dr. Cuartas did, Odele started quietly trimming and simplifying the messiest clauses herself before ever uploading a contract, sanitizing exactly the hard, ambiguous language that was the whole reason the clause was risky in the first place.

The old decision behind it: ClauseKey's team decided not to expose which specific clauses it was least sure about, since "every clause gets the same thorough treatment" sounded like a selling point. That held up fine on short, clean contracts. It stopped holding up once contracts got long enough that difficulty genuinely varied clause to clause.

Same four missing signals, a contract clause standing in for a radiology finding this time.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "hedge the language, border the section, require a named second sign-off, at low confidence," and stop.
Cost: there's no engineering time this quarter to build a full second-reviewer routing system. Say so honestly, and ship the hedged language and visible border alone first, since that costs far less than the routing logic and still stops a reviewer from skimming past.
The model gets better, for real: if ScanDraft's confidence calibration improves and low-confidence cases get rarer, that's still not a reason to remove the flag. It's a reason to trust it more, since a rarer flag is even easier to miss if it ever goes quiet entirely.

Where people run it wrong.
They add a small confidence percentage but leave the surrounding sentence reading just as confident as ever.
They flag the section but route it back to the exact same reviewer who already approved it once.
They treat a near miss as a reason to remove all delegation, instead of fixing the one interface gap that caused it.

How to use it live. When someone asks how the interface should change at low confidence, ask yourself: if I only changed the number and nothing else on the page, would anyone's behavior actually change? If the honest answer is no, the number was never the fix.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0