Define accuracy, usefulness and trust as three distinct measurable properties.

Three words that get used as if they mean one thing. They do not, and mixing them up is how a good tool ships unused, or a risky one ships trusted for the wrong reason.

How to answer this, stage by stage

This question sounds like a vocabulary check. It is not. It is checking whether you know a tool can win on one of these three and lose on the other two, and whether you would catch that before a partner does.

Let's learn

For nine years, Bronwen Sablan pulled every case her firm's litigators cited by hand. Before a brief went out the door, she opened each citation in the legal database herself, read the actual holding, and checked it said what the brief claimed it said. A fifty citation brief took her close to five hours, and in nine years she never once let a fabricated one through.

Then Thorncastle brought in Casewright, a tool that reads a brief and checks every citation the same way Bronwen used to: is the case real, and does it say what's claimed. It runs the same fifty citation check in about ninety seconds. On the firm's own test set of three hundred citations where the right answer was already known, it caught ninety seven out of every hundred planted problems, and it barely ever cried wolf on a clean one.

Ninety seven percent accuracy sounds like the win. It was not the problem, and on its own it was not the fix either. Bronwen kept reopening cases the tool had already cleared, out of habit, out of caution, because a correct answer with no explanation attached is still just a claim. Her total review time on a fifty citation brief barely moved, from five hours down to about four hours and twenty minutes, for the first two months.

At its worst, this costs you twice. You pay for the tool and you still pay Bronwen's hours, because usefulness never caught up to accuracy. Then, five months into using it, a citation Casewright had cleared came up at oral argument. The case was real. It said exactly what the brief claimed. It had also been overruled five months earlier by the state's own supreme court, and nobody, not Casewright, not the golden set that scored it, had ever checked whether the case was still good law.

What I would leave alone: the raw existence check, does this case appear in the reporter at all, does not need touching. It was never the gap. It still runs the same way, still costs almost nothing to compute, and it is still the first and cheapest line of defense against the failure that scared everyone in the first place.

The lesson: a golden set only ever tests the failures you were already afraid of. High accuracy is not proof there is no failure left. It is proof there is no failure left inside the version of the problem someone remembered to write down.

Now here is the same thing as a story

The short version sits above. Read on for the hallway conversation a partner had with Bronwen after a hearing neither of them enjoyed.

Bronwen has run citation checks at Thorncastle for nine years. She can spot a misquoted holding before she finishes reading the sentence, and she knows the firm's Bluebook style the way some people know a phone number they've had since childhood.

When Casewright launched, the product team behind it reviewed its failures every single week. Any near miss a paralegal reported got a new planted test case added to the golden set within days, so the tool's blind spots kept shrinking on purpose.

That habit thinned out in three quiet steps. Once accuracy first crossed ninety five percent, the weekly review became a monthly one, because nothing urgent seemed to be turning up. A few months after that, a near miss report from a different office got logged as "add to golden set later" instead of that same week, since the queue for real feature work was full. By month five, the golden set had not had a new failure category added to it in over five months, right as the live risk quietly shifted from fabricated cases, which Casewright now caught easily, to real cases that had simply stopped being good law.

The trigger was not a crisis. It was a new associate, sitting second chair at a hearing, who leaned over afterward and asked Bronwen a question she could not answer: why had Casewright cleared a citation that opposing counsel had just stood up and called dead law in open court.

The case was real. Doe v. Meridian Trust said exactly what the brief claimed it said. It had also been overruled by the state supreme court five months earlier, and Casewright's golden set, tuned entirely around existence and wording, had no test in it for currency at all. Nobody had lied. The tool had done exactly what it was built to check, and that was the whole problem.

A partner pulled Bronwen aside after the hearing, not angry, just direct: "You're not still trusting that thing blind, are you?" She was not, not entirely, her re-check rate had only drifted down to about eighty percent by then. But eighty percent of citations still meant one in five went out unchecked, and this was the one in five that mattered.

What she did next took three days. She and two associates manually re-audited every citation Casewright had cleared across the firm's nineteen active briefs, close to nine hundred citations in total, forty hours of billable time nobody had planned for. They found exactly one more, a footnote citation in an unrelated matter that had also quietly gone bad.

Here is the old decision, remembered the way you remember a meeting you were in. When Casewright first shipped, the team debated adding a live feed that tracks whether a case has been overruled, appealed, or superseded, the same kind of signal senior litigators call checking whether a case still stands. They decided against it. The feed cost real money to license, adding roughly forty thousand dollars a year, and it made each citation check slower, about four extra seconds apiece across a whole brief. The fear that justified skipping it was fabrication, because that was the story that had scared every firm that year. Currency was not on anyone's mind, because nothing had gone wrong with it yet.

I would take that decision back, but not everywhere at once, because the extra cost and the extra four seconds are real. I would ship the currency check first on the categories of law that move fastest, constitutional and regulatory citations, where a case going dead is common, and leave the slower moving areas, like well settled contract doctrine, on the cheaper existence only check.

This is the whole gap in one picture. The golden set never asked the second question, because the failure it was built for never needed it to.

Run the same nineteen briefs again with the fixed golden set in place. Casewright flags Doe v. Meridian Trust itself, the week the overrule was published, before a single brief citing it is ever filed. Five months of exposure and one hallway conversation become zero.

What I would tell myself, back in that first launch meeting: do not build your golden set only out of what already scared you. Build it out of what a high accuracy score will eventually stop you from noticing.

LEAD, run on three properties instead of one number

Not a diagnosis of one bad citation. LEAD run on the question itself, using accuracy, usefulness and trust as the three things being linked, signaled, gamed and decided on.

Three things worth stating plainly, since this is where the real judgment sits. The rejected alternative that mattered most was the live currency feed at launch, a real and defensible call given what the industry was actually afraid of that year, not a mistake made carelessly. The AI-specific failure worth naming by name is a form of distribution shift inside the eval itself: a golden set tuned for one failure family, fabricated citations, will keep scoring near perfect once a different failure family, stale but real citations, becomes the live risk, because nothing in the test set was ever built to catch it. The guardrail is concrete: refresh the golden set on a schedule against a live subsequent-history source, not just when a paralegal happens to report a near miss, and gate the more expensive currency check to the categories of law where it moves fastest. The trade-off is real and was accepted on purpose: checking currency costs about four extra seconds per citation and roughly forty thousand dollars a year in licensing, paid first where the risk is highest, not everywhere at once.

And if you want to be sure it really works, try it somewhere else

Same four letters, a community pharmacy instead of a law firm, and no citations anywhere in sight.

Dosewatch is a tool that checks a new prescription against a patient's other active medications and flags a dangerous interaction before the pharmacist fills it. Perrine Ledbetter runs the pharmacy counter where it was rolled out first.

L, link. What the pharmacy chain actually wants is for Perrine to stop manually cross-referencing every refill against a paper interaction chart, without a single preventable interaction ever reaching a patient.
E, early signal. Dosewatch's accuracy against a golden set of known interaction pairs held near ninety nine percent for months. It moved first, and by itself it said nothing about whether Perrine had changed a single thing about her counter routine.
A, abuse. The golden set was built from interaction pairs already published in reference databases, so it stayed accurate for a newly published interaction it had simply never been tested against, an accurate tool blind to the exact case that mattered this quarter. On the usefulness side, the easiest way to game an "accepted the flag" rate is to suppress the borderline ones, so only the obvious flags remain, and acceptance looks great while real coverage quietly shrinks.
D, decision. If accuracy lags on newly published interactions, hold Dosewatch to a supervised mode for any drug approved in the last two years. If accuracy is fine but usefulness lags, replace the confidence percentage with the actual mechanism, which enzyme pathway is involved, since a number without a reason rarely changes a pharmacist's habit. If trust is climbing suspiciously fast, check whether it is arriving because Dosewatch earned it or because a short staffed shift left nobody time to double check anything.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to naming the three as separate measurements and say which one you would ask about first, before defining any of them in detail.
Cost: a live "still good law" or "newly published interaction" feed is expensive. That is not a reason to skip it everywhere, it is a reason to gate it to the highest risk slice first and say so out loud.
The model got better, for real: accuracy genuinely climbs to ninety nine point five percent. Say plainly that a higher score on the same golden set still does not test a new failure family, and does not move trust on its own.

Where people run it wrong.
They treat accuracy as trust and never build a way to measure usefulness at all.
They measure usefulness by asking users if they like the tool, a self-report number, instead of watching what those same users actually do.
They chase trust by asking people to rate their own confidence, instead of watching a real behavioral proxy like the re-check rate.

How to use it live. Open by naming that these are three different measurements, before defining any single one of them. That one sentence buys you the room to answer slowly and correctly instead of rushing into a definition that quietly blends all three into one.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0