Describe a product decision you would make differently if the model's error rate doubled overnight.

Emberpitch is Quillmarch's campaign generator: give it a business's product brief and it writes back ad headlines, body copy, and a week of social posts and email subject lines. Elowyn Prevette owns Emberpitch's quality numbers. Quillmarch swapped in a cheaper, faster base model to cut inference cost, and the next morning's eval showed the claim-error rate had basically doubled. Iwalani Desroches, who runs brand and legal risk, needed a decision before that day's campaigns went out.

How to answer this, stage by stage

Nobody's grading whether you can say the error rate doubled and sound concerned. They're grading whether you can turn that into a number of wrong lines a day, and a decision that survives someone asking where 132 to 198 actually came from.

Let's learn

Every Tuesday morning, Elowyn Prevette pulls the same 600-line sample and checks one number. For months, it barely moved.

The number is Emberpitch's claim-error rate, how often a generated ad line gets a fact wrong. Emberpitch is Quillmarch's campaign generator: a small business owner describes what they sell, and it writes back ad headlines, body copy, and a week of social posts and email subject lines, ready to publish.

Before last month, Elowyn's number was reassuring. About 20,000 of Emberpitch's 57,000 daily lines carry a claim, a number, a guarantee, a certification, a comparison, the kind of line that can be factually wrong. At a 2.1 percent error rate on those, that's 420 wrong lines a day, comfortably under the 2.5 percent line Quillmarch had set for its most reliable customers to skip review entirely.

Five numbers hiding behind one word, doubled. Say what each one is before reacting to any of them.

About 300 of Quillmarch's 2,300 accounts, its largest and longest-standing customers, had graduated out of manual review months earlier. The model had been steady long enough that a person checking every line felt like busywork nobody needed.

Then Quillmarch swapped in a newer base model, faster, cheaper to run, better at punchy hooks. Overnight, the next weekly eval read 4.3 percent. Not a drift. A jump.

Two minutes apart. Same dashboard, same chair, one number that flipped a whole morning.

Iwalani Desroches, who runs brand and legal risk at Quillmarch, doesn't see a percentage. She sees a landscaping company's ad claiming a certification it doesn't hold, live, published, running, because the account graduated out of review back when 2.1 was the number everyone trusted.

What it costs at its worst is more than a correction email. A fabricated certification or an invented discount sits under a real small business's name, on a platform that can suspend an entire ad account for one unsubstantiated claim, not just pull the one line. The business that hired Quillmarch to save them time ends up spending it explaining to a customer, or a platform reviewer, why their ad said something nobody at the company ever claimed.

What I would leave alone: the other 65 percent of Emberpitch's output, generic hooks, tone variants, emoji placement, calls to action with no claim in them. A doubled error rate there costs a business owner a slightly flatter headline, not a false claim with their name on it. Gating those the same way just slows down copy that was never the risk.

The lesson: the model didn't fail. Quillmarch built an exemption around a snapshot instead of a standing check. A number is only safe to build an exemption on the day it's treated as a range you keep watching, not a fact you file away.

Now here is the same thing as a story

Read the short version above when you're in the room. Read this one when you want to feel why a doubled number is never really about the number.

For eleven months, the worst part of Elowyn Prevette's Tuesday was deciding what to have for lunch.

She runs quality for Emberpitch, Quillmarch's campaign generator, and every Tuesday she pulls a 600-line sample of claim-bearing copy, a number, a guarantee, a certification, a comparison, and has a person check each one against the business's own brief. For eleven months the number held steady around 2 percent. Good enough that, one by one, Quillmarch's biggest accounts got moved off mandatory review and onto auto-publish. Elowyn signed off on each one. The number said it was fine.

Then the infrastructure team shipped a new base model. Faster. A third cheaper to run. Reviewed for tone and hook quality, and it tested well, funnier headlines, tighter hooks, the kind of copy that makes a small business owner actually want to post it.

Nobody reviewed it for claims. Nobody thought to. The old model hadn't been a claims problem in almost a year.

Elowyn pulled Tuesday's sample the way she always did, coffee first, dashboard second. 2.1 last week. She scrolled. 4.3 this week.

She read it twice.

She didn't panic, and she didn't wait for a bigger sample either. She called Iwalani Desroches, who runs brand and legal risk two floors down, before she'd finished her coffee.

Iwalani pulled up a handful of that week's flagged lines while Elowyn talked. A pressure-washing company's ad claiming a certification the company had never held. A landscaper's headline promising a discount that didn't exist in the brief. Small, specific, wrong, and every one of them auto-published, live, under a real business's name, because the account had graduated out of review back when 2.1 was the number everyone trusted.

This is the fork every extra wrong line hits. The graduated accounts skip the left branch entirely.

Here's the part that isn't really about the model. Elowyn had a number in her head that only had two settings: under the line, or over it. She never carried the actual percentage around, she carried a feeling, calm on a Monday, not calm the moment 4.3 showed up. Two point one to four point three flipped the feeling in one look at a spreadsheet.

So here's the decision Elowyn and Iwalani made that morning, before the second coffee.

Months earlier, when the auto-publish program launched, the rule had been: hit 2.5 percent or under for a full quarter, and the account graduates out of review, for good. Nobody built in a re-check. Nobody asked what happens if the model underneath changes and the account doesn't know to ask again.

I would take that back. Not the graduation itself, the businesses that earned it deserved faster copy. I'd take back making it permanent. Tie it to a live number instead: the moment the weekly eval crosses 2.5 percent on claim-bearing lines, review comes back automatically for every graduated account, no waiting for someone to notice.

Run the same Tuesday again with that rule in place. The eval comes back at 4.3. The system, not Elowyn, pulls the 300 graduated accounts back into the review queue before the day's first campaign goes out. About 440 extra wrong lines get generated that day either way, the model's rate is the model's rate, but somewhere between 132 and 198 of them, the ones that would have shipped straight to a live ad, get a person's eyes on them first instead.

One design waits for a person to notice a spreadsheet before anyone with a lever is told there's a problem. The other design tells the lever the moment the number crosses the line.

What I'd tell my past self, the one who set the graduation rule as a one-time pass: a number that earns someone an exemption today is not a promise about the number tomorrow. Build the check to keep watching, not to file the result away.

BOUND, or doing the arithmetic before the campaign goes live

Not a way to sound more careful about a doubled number. BOUND is what turns "the error rate doubled" into a count anyone can act on, and a decision that survives someone asking where the numbers came from.

All four numbers on the same scale. 4.3 doesn't sit near the line, it sits well past everything on it.

The gate's catch rate sits exactly where a good estimator worries most: least grounded, most consequential.

The whole decision in five boxes. The fourth box is the one that turns a number into an action.

Three things worth naming directly, since this is where the real judgment sits. The AI-specific failure mode is a plain hallucination: the new model invents specifics, a certification, a percentage, a limited-time discount, that were never in the business's own brief, because it's optimizing for a punchy line over a grounded one. The guardrail today is the confidence gate; the guardrail worth building next is a check that verifies every claim-bearing line against the fields in the business's actual brief before it's allowed to publish, not just a confidence score. The alternative I rejected is rolling the whole model back overnight, tempting because it looks like the safest move, but it throws away real gains on everything that isn't a claim and takes longer to redeploy than a gate setting takes to flip. And there's a real trade-off in the decision that shipped: tightening the gate means campaigns that used to auto-publish in under a minute now wait for a person, sometimes hours, and Quillmarch's review team costs more hours a week doing it. That's the price, paid in speed and headcount, for not paying it in fabricated claims sitting under a real business's name.

And if you want to be sure it really works, try it somewhere else

Same five letters, a translated appliance manual instead of an ad. This time the rare thing isn't a fabricated discount. It's a mistranslated voltage warning.

Same method, a different label on the box. Both times, the graduated exemption was built on a number that quietly stopped holding.

Marrowdene Appliances ships washing machine and water heater manuals in fourteen languages through Verityscript, a translation tool that also checks safety-critical lines, voltage warnings, chemical handling steps, against a certified terminology glossary before a manual goes to print. Osmund Faircloth owns Verityscript's numbers. Marrowdene swapped its translation model for a cheaper one to cut localization costs, and the next batch's safety-line error rate, measured against the glossary, roughly doubled overnight: 1.8 percent to 3.6.

Osmund runs the same five letters. Break it down: about 50,000 safety-critical lines ship a month, across 40 manuals in 14 languages, and the number that matters isn't the manual-wide average, it's the error rate on lines a certifier actually inspects. Own the numbers: at 1.8 percent, that's 900 wrong safety lines a month before the swap, 1,800 after, about 900 extra. Use a range: bilingual reviewers spot-check and catch an estimated 60 to 75 percent before print, an estimate, not a full count, so somewhere between 225 and 360 of those extra wrong lines reach a printed manual. Nail the sanity check: the certification body's own tolerance for mistranslated safety-critical text is under 1 percent, and even the 1.8 percent baseline, before anything doubled, was already over that line. 3.6 percent is nearly four times it. Direction: same lever as Emberpitch's story, the swing assumption is the reviewer catch rate, the least grounded number in the equation and the one that moves the estimate most if a spot audit turns out to be optimistic.

Same rank, different lever: the fix for Verityscript isn't a bigger review team either. It's the same habit Elowyn ran on Emberpitch's numbers: pull the risk-bearing category out of the average, range the number that's genuinely uncertain, and route that category, specifically, back to a human, regardless of what the aggregate error rate says.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: claim-bearing volume, times the error-rate delta, times what the gate actually catches, times how often a miss gets noticed, checked against a real threshold.
Cost: no budget to add a grounding step this quarter. Keep the wider human-review net on the graduated segment as the interim cost, it's cheaper than one recalled campaign or one denied ad account.
The model got better, for real: say the next version's claim-error rate came in at 1.0 percent instead of 4.3, half the old rate. The graduated segment doesn't need widening either, it needs the same threshold check, just checked in the other direction, before anyone loosens review somewhere else.

Where people run it wrong.
They report the raw doubling as the finding and stop there, without turning it into a daily count anyone can act on.
They test the whole product's error rate instead of isolating claim-bearing lines, and dilute a real signal into noise.
They treat the confidence gate's catch rate as a fixed fact instead of an estimate from spot audits, and build a false sense of precision on top of it.

How to use it live. Ask "what's the volume this actually happens on, and what already catches it?" before quoting a percentage. That's the two multipliers a doubled rate needs before it means anything.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0