What does 'correct' mean for a summarization feature? Give a definition your engineering team could test against.

Threadmark is Quillhorn's feature that turns a long customer support ticket into a three-line summary, so an agent picking up someone else's case doesn't have to read the whole thread first. Pallavi Onuoha owns Threadmark's product story. Baldassare Belden, the staff engineer who builds it, keeps getting handed a spec that says the summary should "capture the key points." He can't write a test for that sentence, and three weeks ago that gap let a made-up refund reach a real customer.

How to answer this, stage by stage

Nobody is grading whether you can say "we'd write clear acceptance criteria." They're grading whether you can hand an engineer something they could actually put in a test file.

Let's learn

What happens the first time two people read the same AI summary and can't agree on whether it's right?

Threadmark is Quillhorn's feature that reads a customer support ticket, the whole back-and-forth thread, and hands an agent a three-line summary instead of the full history.

Before Threadmark, this was the whole job. Every reassigned ticket, read from the top, every time.

Before Threadmark, an agent who picked up a reassigned ticket, one handed off from someone else, had to read the full thread first. The average reassigned ticket runs 14 messages back and forth. Reading it before replying took about four minutes. Quillhorn's floor handles around 30 reassigned tickets a day.

With Threadmark, that same agent reads a 45-word summary in about 20 seconds and replies. Multiplied across 30 tickets a day, that is real time back, and agents noticed immediately.

Here's the turn. Three weeks ago, Quillhorn's engineering team swapped the model behind Threadmark for a cheaper, faster one. Before shipping it, a reviewer read 20 sample summaries. All twenty looked fine. That is not really the problem. The problem is that "looked fine" was never a test anyone could point to, so nobody could say what would have made a summary fail.

What it costs at its worst: a fabricated fact that sounds confident is worse than an obviously bad summary, because nobody double-checks a summary that reads well. If Threadmark keeps quietly inventing facts and nobody can test for it, agents eventually stop trusting any of it and start re-reading full threads again, on top of the summary they no longer believe. That erases the time saved and adds doubt where there used to be none. Worse than never shipping Threadmark at all.

What I would leave alone: Threadmark also suggests a one-word tag for each ticket, like "billing" or "shipping," that an agent can change with one click. That doesn't need this rubric. A wrong tag costs three seconds to fix and never reaches a customer. Only the parts of the output someone might act on without checking need a real test.

The lesson: an untestable spec doesn't fail loudly. It fails the first time someone trusts it, quietly, in a sentence that sounds completely normal.

Now here is the same thing as a story

Read the short version above when you're in the room. Read this one when you want to feel why a sentence nobody could test was the actual bug.

The rubric card taped above Baldassare Belden's monitor had four lines on it, and by the time this story starts, he'd written all four himself.

Baldassare had built the model layer behind Threadmark since it was a prototype that only worked on billing tickets. Pallavi Onuoha, who owned the product, wrote the original spec ten months ago: the summary should capture the key points of the conversation, in plain language. It sounded right. Nobody in the room disagreed with it. Nobody could have written a test for it either, and at the time that didn't matter, because Pallavi read every single summary herself before it shipped.

For the first few months, that was fine. Threadmark launched, agents loved it, and the four-minute read on a reassigned ticket dropped to twenty seconds. Zuriel Muya, who runs the support floor, started routing more reassigned tickets through the same-day queue instead of holding them, because agents no longer dreaded picking one up cold.

This is the chain nobody meant to build. Every step looked reasonable on its own.

As Threadmark grew past billing into every ticket type on the floor, Pallavi stopped reading every summary. She couldn't; volume made that impossible. The job moved to a rotating reviewer who read 20 random summaries a week and rated them, informally, "looks fine" or not. For a while, they always did.

Three weeks ago, engineering swapped the model behind Threadmark for a cheaper one, call it Model B, to cut inference cost. The reviewer that week read 20 summaries. All twenty read cleanly: short, confident, plain language, exactly what the spec asked for. It shipped.

Two weeks ago, a customer wrote in twice about a delayed order. In the first message, they said, "if this isn't fixed by Friday I'd like a refund." A different agent, working an unrelated case for the same customer, had separately noted an $86 credit applied to a different order entirely. Model B's summary merged the two: "Refund of $86 already processed for this order." It hadn't been. A support agent, reading only the summary, told the customer the refund had gone through.

Same fabricated line. One definition of correct lets it through. The other stops it before an agent ever sees it.

The customer escalated, and the case landed on Zuriel's desk. It wasn't the $86 that worried her most. It was that three agents, when she asked, said some version of the same thing: they'd started re-reading full threads on their trickiest tickets again, just in case, because they could no longer tell which summaries to trust.

Baldassare pulled the thread and found the merge in about ten minutes. There was nothing subtle about the bug once you looked. The hard part wasn't finding it. The hard part was that nothing had been checking for it in the first place, because "captures the key points" gives an engineer nothing to assert against.

Same model, two ways of looking at it. One of them was never going to find this.

What Pallavi did next wasn't add another reviewer, or ask people to read more carefully. She sat down with Baldassare and Zuriel and wrote a real definition. Not "captures the key points." A rubric, run against 240 real, redacted tickets with human-written reference summaries: zero invented facts about money or status, every remaining claim traceable to a real line in the thread, entities preserved, under 60 words. Baldassare wrote it as an actual test suite that afternoon.

Four lines, taped above a monitor, that finally give an engineer something to write code against.

Run the same three weeks again, with that rubric already gated into the release process. Model B still gets built. This time, before it ships, it runs against all 240 tickets. Eleven of them fail the zero-tolerance line, including the exact merge that caused the $86 refund. The release is blocked, Baldassare fixes the entity-linking bug that let facts bleed across notes, and Model B ships two days later at zero fabricated claims on the same set.

The decision Pallavi would take back traces all the way to that first spec, ten months earlier. "Captures the key points" made sense the day she was reading every summary herself and could catch anything off by eye. It stopped making sense the day review became a 20-summary skim on a product handling thousands of tickets a week, and nobody had noticed the sentence needed to become a test.

What she'd tell herself, back at launch: a definition that only a person can judge isn't a definition. It's a placeholder for one, and it will hold up right until the day nobody's watching closely enough to notice it never became real.

SPARK, in one screen: the definition Baldassare could actually test

Not a way to make "captures the key points" sound more official. SPARK forces the actual rubric you'd hand an engineer, and makes you prove it survives being wrong in both directions.

Three rows that ship day one. One row, deliberately, that doesn't yet.

And if you want to be sure it really works, try it somewhere else

Same five letters, a veterinary group instead of a support desk, and this time an invented dose instead of an invented refund.

Briarcombe Veterinary Group runs Vetlume, a feature that summarizes a pet's full visit history for the vet before an appointment, pulling together notes from every past visit, sometimes across different vets, into a short brief. Dr. Estelline Sathe, who leads clinical product for Briarcombe, inherited the same kind of spec Pallavi once wrote: the visit summary should "capture what the vet needs to know." A partner clinic asked her, after a near miss, what "correct" actually means for that summary, and whether it's something they could test before the next model update ships.

Different vertical, same shape of danger: a fluent, confident claim that never happened.

Same rank, different lever, mapped straight onto SPARK: the situation is the same gap, a spec nobody can test, this time on medication history instead of refund status. The payoff is the same habit: a vet trusts the summary because it's been checked, not because it reads confidently. The anchor is the same shape: a golden set of real, redacted visit histories, zero tolerance on any invented dose, diagnosis, or allergy, every other claim traceable to an actual chart entry, under a length cap. The risk is the same two directions: too loose lets a wrong dose slip through; too strict fails a summary for paraphrasing a diagnosis and gets ignored. And what stays out is the same discipline: whether the summary "reads warmly" isn't graded yet, because it isn't what could hurt an animal.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: zero tolerance on invented facts that could cause real harm, everything else scored against a real, checked set, gated before ship.
Cost: no budget to build a 240-ticket golden set this quarter. Ship a smaller one, 40 tickets, on the highest-risk category only (refunds, or medication), and grow it as budget allows, rather than shipping no test at all.
The model got better, for real: say the new model's fluency jumps and reviewers say every sample "reads beautifully." The rubric doesn't relax. A better-sounding model is exactly the one most likely to fool a skim-based review, so the fact-check gate matters more, not less.

Where people run it wrong.
They write a rubric with no zero-tolerance line at all, so a fabricated fact just lowers a score instead of blocking a release.
They grade paraphrase mismatches as strictly as fabricated facts, so the gate cries wolf and gets silently overridden.
They build the golden set once at launch and never touch it again, so it stops covering whatever the product does two years later.

How to use it live. Before naming a number, ask the coverage question: "What's actually been checked, on real examples, and what's still just a reviewer's impression?" That question alone usually shows whether a "definition of correct" is a real test or a vibe with a rubric-shaped name.

Three things worth stating directly, since this is where the real judgment sits. The alternative Pallavi's team considered and rejected was a single holistic LLM judge, one prompt asking "is this summary good, yes or no." It lost, because early testing on the incident showed it rated the $86 hallucination "yes, good" nine times out of ten; a fluent, confident sentence reads as good to a judge that isn't specifically checking whether each claim traces back to the source. They also tried scoring against a reference summary with plain word-overlap. That failed the same way from the other direction: the fabricated refund line reused real ticket vocabulary and scored well on overlap, while an accurate summary that paraphrased the resolution scored lower despite being right. Neither method was testing the thing that actually mattered. The AI-specific failure worth naming is cross-message fact bleed: a model merging content from two unrelated notes about the same customer into one confident, wrong claim, a distribution the model had never been explicitly taught to keep separate. The guardrail is the traceability check itself, refusing to accept a claim that can't be pointed at a specific line in that one ticket's thread. And the trade-off is real: running the full fact-check pass against all 240 tickets, for every candidate model, costs meaningfully more time and compute than a 20-sample skim, so the team accepted running the full battery only at the pre-launch gate, a few times a week, instead of on every commit. Slower feedback, in exchange for a signal worth trusting.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0