Write the severity definitions for AI quality incidents.

How to answer this, stage by stage

Nobody is grading whether you can name four severity levels. They're grading whether your ranking rule survives a case where the rare error is the dangerous one and the common error is the harmless one. Six moves get you there.

Let's learn

Say we build a tool that listens to a doctor and patient talking, then drafts the visit note before the doctor leaves the room. Before a tool like this existed, doctors typed or dictated every note themselves, thorough, but it ate into evenings and weekends that should have belonged to their own lives.

Then Silvermere Health rolled out Fieldnote. About 900 doctors across fourteen hospitals started using it, and the average note went from twelve minutes of typing to about ninety seconds of review and sign-off. Doctors stopped writing notes from scratch and started reading a draft, fixing what needed fixing, and signing.

Now say plainly what the mistake was. The first severity policy Silvermere wrote ranked incidents by how wrong the model's own confidence score said a note was. A low-confidence note got flagged high severity. A high-confidence note, even a fabricated one written in a calm, fluent, medically plausible voice, got treated as low risk, because the model sounded sure of itself.

At its worst, a fabricated allergy sits in a structured field, gets signed at the end of a fifteen-minute visit, and reaches the pharmacy's order-checking system before any human reads it as a sentence. A wrong plural in the assessment paragraph, by comparison, sits in front of the doctor's own eyes for the entire ninety seconds of review. One of these gets caught by a person almost every time. The other one might not.

Before any model update to Fieldnote ships, it has to clear a bar on a fixed golden set, 200 known structured-field facts pulled from real chart reviews, catching close to all of them, not a promise that it will always be right on every note. A model that clears that bar can still produce a fabrication in production. That is what a 99 percent bar looks like at scale, by design, and the tiered response is what protects the patient on the days it happens.

The choice I would take back. When the incident policy first got written, severity was scored off the model's own word-level confidence, an easy number to pull straight from the vendor's API, and it looked rigorous on a dashboard. It stopped making sense the moment the field-level audit showed the rare, quiet errors in structured fields were the ones nobody was reading before they acted.

There was a cheaper-looking fix on the table too: require a clinician to manually confirm every structured field before it can sync anywhere, on every single note. That would push the Sev-1 rate close to zero. It would also give back nearly all of the time Fieldnote was built to save, since the ninety-second review would balloon back toward the twelve minutes doctors used to spend typing. Silvermere didn't take that trade. It accepted a small, audited residual error rate in exchange for keeping the fast review, and put the safety margin into same-day escalation instead of upfront friction.

What I would leave alone. A wrong verb tense, an awkward sentence, a slightly clumsy summary in the narrative paragraph, none of that belongs on the incident scale at all. It costs a doctor two seconds to fix while reading, and folding it into the same ladder as a fabricated allergy just buries the tier that actually needs same-day attention.

The lesson. A severity scale built around how wrong something sounds will always underrate the error nobody gets a chance to hear.

Now here is the same thing as a story

The short version is above. Read on if you want to feel how close the confidence-score policy came to missing the one that mattered.

Yordanos Bekele ran the emergency department at a Silvermere hospital for nine years before she moved into product. She still reads a chart the way a charge nurse does, looking for the one line that changes what happens next, not the line that looks the messiest.

Fieldnote had been live for fourteen months when the incident policy Yordanos inherited still scored severity off the model's own confidence number. It was simple, it was already in the API response, and for most of that year it worked fine, because most errors that showed up were exactly what you'd expect: a garbled sentence here, a misheard word there, always sitting in the narrative paragraph, always caught by the doctor reading it before signing.

Then, on an ordinary Thursday afternoon, a resident finished a fifteen-minute visit, skimmed the Fieldnote draft, and signed it. The structured allergy field now read "penicillin, confirmed by patient." The patient had never said that. Fieldnote had picked up a stray mention of a cousin's penicillin reaction earlier in the recording and attached it to the wrong person.

The model's confidence score on that note was high. The sentence was fluent, specific, and calmly worded. Nothing about it looked like an error on a dashboard.

The chart was already signed. The allergy field had already synced to the pharmacy's order-checking system. A nurse pulling up the patient's chart twenty minutes later to prep a follow-up prescription saw the allergy, paused, and asked the patient directly. The patient said she'd never had a reaction to anything. The nurse caught it by asking one extra question she didn't strictly have to ask.

A narrative typo gets fixed the moment someone reads it. A signed structured-field entry is already acting on its own by the time anyone reads it as a sentence.

Yordanos pulled the incident log from the month before. Every flagged incident that quarter had been a narrative-text error, the kind a doctor fixes in the two seconds it takes to read a sentence. Not one structured-field error had ever been flagged, not because none had happened, because the confidence score never once dipped low enough to trip the policy's threshold on any of them.

She asked the informatics team to pull 1,200 signed notes and check every field against the actual visit recording. Structured fields, the ones pharmacy and order sets read directly, carried a fabricated or wrong entry in 0.4 percent of notes, five notes out of the twelve hundred. Plain narrative text carried an error in 3.1 percent, thirty-seven notes. The narrative errors were seven times more common, and every one of the five structured-field errors had scored high confidence.

What stayed with Yordanos was how close the original policy came to never flagging a single one of the five, quarter after quarter, simply because a fabricated sentence written with confidence looks exactly like a correct one to a score that only measures how sure the model is of its own words.

The redesigned policy now scores severity off two questions, not one: does this field feed an automated action with no person checking first, and did that happen before or after a clinician's own review. A fabricated allergy caught before sign-off is Sev-2, urgent, but reversible. The same fabrication reaching pharmacy after sign-off is Sev-1, same-day escalation, model rollback considered, and a same-day chart correction sent to every downstream system that already read it. Run that quarter's five structured-field errors back through the new policy and all five get flagged the day they're signed, hours after sign-off at worst, instead of one of them surfacing by chance because a nurse happened to ask an extra question.

The thing Yordanos would tell her past self, back when that first policy shipped: a number that tells you how sure the model felt was never going to tell you what the sentence was about to do.

ORDER, the five things behind the tiers

This is a ranking question, what makes one AI documentation error worse than another, not a story about a person's habit switching between two settings, so ORDER fits and FLIPS doesn't.

O, outcome. What every tier is competing to protect: whether a wrong or made-up clinical fact reaches a point where it drives an action on the patient before a human gets a chance to catch it. Not "how wrong the model was," a generic accuracy question that says nothing about consequence.
R, reversibility. Rank by how hard the resulting harm is to undo. A structured-field fabrication that's already synced to pharmacy is far harder to undo than a narrative typo a doctor fixes while reading, even though the typo shows up seven times more often.
D, dependency. Before any tier means anything, you need a map of which fields feed an automated downstream action with no person in between. Silvermere didn't have that map on day one; it had a confidence score, which measures something else entirely.
E, evidence. The 1,200-note field-level audit: 0.4 percent error rate in structured fields, 3.1 percent in narrative text, and every one of the five structured-field errors carried a high confidence score. Cheap to run, and it's what actually located the real Sev-1.
R, rank. Sev-1: fabricated or wrong fact in an automated-reading field, after sign-off. Sev-2: the same kind of fact, caught before sign-off. Sev-3: a wrong or missing narrative-only detail. Sev-4: formatting or wording, no factual error. Defend the top pick in one line: Sev-1 outranks everything else because it is the only tier where harm can reach the patient before the hospital knows an error happened.

Each box has to happen before the next one is trustworthy. Skipping straight to ranking without knowing who reads the field is how a confidence score ends up standing in for consequence.

And if you want to be sure it really works, try it somewhere else

Amberline Power runs an AI tool that drafts outage and incident reports for its control room from radio chatter and sensor feeds, so dispatchers don't have to type up every call while a storm is still active.

O, outcome. What every tier competes to protect: whether a wrong report sends a repair crew somewhere unsafe, or clears an outage the public still can't see, before a dispatcher reviews it.
R, reversibility. A draft report that auto-populates the public outage map is far harder to undo once residents have already acted on it than a report still sitting in a dispatcher's review queue.
D, dependency. Amberline's PM, Talise Aluko, first had to map which report fields feed the public map and the crew dispatch queue automatically, versus which fields only a human dispatcher ever reads.
E, evidence. A review of 400 draft reports found the AI wrongly marked "no live wires down" in 0.5 percent of cases, always in the auto-published field, and mislabeled a minor equipment code in 4 percent of cases, always in a field a dispatcher edits before anything goes out.
R, rank. Sev-1: a wrong live-wire status reaching the auto-published map or a crew dispatch, before dispatcher review. Sev-2: the same error, caught first. Sev-3: a wrong equipment code in dispatcher-reviewed fields. Sev-4: report formatting.

Swap the trigger and it still runs.
Speed: an interviewer caps the answer at ninety seconds. Skip straight to the rule: rank by whether a field feeds an automated action with no person checking first, not by how often the error happens or how wrong it reads.
Cost: the informatics team can only run a small audit this quarter. Don't skip mapping which fields are automated to save time, that map is what makes every tier after it mean anything; shrink the sample size of the audit instead.
The model got better: Fieldnote's structured-field error rate drops to near zero next year. The four-tier scale doesn't get retired, it just gets checked less often, because the ranking rule, not the current error rate, is what makes it correct.

Where people run it wrong.
They rank severity by how often an error type happens, so the common, harmless one crowds out the rare, dangerous one.
They rank severity by the model's own confidence score, which measures how sure the model sounded, not what happens after.
They fold cosmetic wording issues into the same incident scale as patient-safety errors, so the Sev-1 queue drowns in noise nobody needs to treat as urgent.

How to use it live. Say the rule before naming a single tier: "severity here depends on how hard the damage is to undo, not on how wrong the text sounds, and that comes down to whether something acts on the error before a person reads it." That earns you the room to ask what actually gets automated in this specific product, instead of reciting four made-up labels.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0