At what point does latency stop mattering and quality take over?

Latency stops mattering the moment a finding crosses from a mistake a human would probably catch anyway into one only the model was ever going to see.

How to answer this, stage by stage

Nobody is grading whether you can name a millisecond target. They are grading whether you can say, out loud, which mistakes you're willing to let a human catch instead, and which ones only get caught here.

Let's learn

Picture a tool that sits inside a company's pull request flow. Before a person opens the diff, it reads the change and posts inline comments right on the lines that need one: a variable named badly, a missing test, a permission check that quietly lets the wrong account see the wrong data.

Before a tool like this, a senior engineer read every pull request by hand. About thirty five minutes a PR: reading the diff, tracing the logic, checking whether a change in one file quietly broke an assumption three files away.

With the tool, a first pass lands in about twelve seconds, on every pull request, no matter what kind of change it is. A human still checks it, but that check drops to about nine minutes.

Here is the turn. Marginale being wrong once in a while is not the problem. What engineers do after eight months of watching it move fast is the problem: they stop rereading a hunk themselves the moment Marginale stays quiet on it, and quiet starts to mean clean, even on the part of the diff Marginale never actually finished checking.

At its worst, one shared clock for every comment means the tool never gets the room to look closely at the one kind of change that actually needs it. On a diff that touched a tenant scoped export endpoint, the twelve second clock ran out before a trace across three files could finish, so it posted nothing. The PR merged with nothing to flag it. Eleven days later a customer's own security team noticed a contractor account could pull another tenant's export file just by editing a URL. About ninety hours went into cleanup: engineering, security, and a call with the customer that ended with one contract not renewing.

One of these costs is small and everyone watches it happen. The other one is quiet, and nobody is watching for it on purpose.

What I'd leave alone: the fast lane itself. Keep nits, naming, and missing test nudges landing in about ten seconds no matter what. Most of them really are cheap to get wrong, an engineer skims a naming comment and moves on either way, so slowing that down to match the careful lane would only cost flow with nothing bought back.

The lesson: a clock that's the same for every kind of mistake isn't neutral. It's a bet that every mistake costs about the same to miss, and that bet is almost never true. The real question was never how fast Marginale should be. It was which mistakes the team was willing to let a human catch instead, and which ones only get caught here.

Now here is the same thing as a story

Read the short version above for the two minute answer. Read this one for the week "twelve seconds, no exceptions" still sounded like the safest number in the room.

Kirsi Bakshi spent six years writing backend code before she moved into product, the kind of engineer who could smell a bad permission check three lines into a diff, before she'd even opened the file it called into.

Thackray Systems brought her on to run quality when they built Marginale, the tool that would leave that first pass of comments in her place. The good months were good. Every morning, comments were waiting on a PR before the coffee finished brewing, and a review queue that used to sit until after lunch was empty by ten.

Then the habit changed, in three small steps. First, engineers stopped reading their own diff before pushing it, since Marginale would catch the obvious stuff anyway. Second, they stopped rereading a hunk Marginale stayed quiet on, treating quiet as reviewed and clean instead of not fully checked. Third, reviewers started waving a PR through the moment Marginale posted nothing at all, without ever asking whether Marginale had actually had time to look.

The trigger wasn't small this time. A customer's own security team found that a contractor account, one tier down from full admin, could open another tenant's export file just by editing the URL in the browser. Marginale had been looking right at that endpoint eleven days earlier. It just hadn't had time to finish the trace before its twelve second clock ran out, so it said nothing, and nothing read as fine.

Kirsi didn't have a clean answer when the postmortem asked why nobody caught it. The honest one was that Marginale's launch pitch had been "every comment in under fifteen seconds, no exceptions," set in a planning meeting eight months earlier because that was the number that got engineers to actually install the thing. Nobody in that meeting asked what would happen the day a finding needed longer than fifteen seconds to be sure of.

The decision that opened the door went back to that same meeting. Someone asked whether some checks should be allowed to run longer, and the room agreed that splitting the clock would slow adoption right when they needed engineers to trust the tool. So one clock, for everything, was the safer sounding choice, the week nobody had any track record to lean on yet.

Run the same diff again with one change: a rule, written down before Marginale ever posted a comment, that a finding touching auth, data scope, or the final merge gate gets its own clock, up to about four and a half minutes, while nits keep their ten seconds. Under that rule, the trace on the export endpoint finishes at four minutes and forty seconds, and it posts a comment that blocks the merge until a human looks. The bug never ships. The ninety hours of cleanup never happens.

One design trusted a single number nobody had thought to split. The other design trusted a rule that already knew which mistakes were allowed to take longer.

What I'd tell Kirsi's team, back in that first planning meeting: the day you set one speed for every kind of mistake, you haven't made the product simple. You've hidden the one mistake that needed more time inside all the ones that didn't. Nobody in that room meant to do that. That is exactly how it happens.

PICK, four moves for where the clock should stop

Not a rule for making everything fast or everything careful. Four moves that build toward the one line that actually decides it: which kind of wrong you can afford to let ride, and which you can't.

Three things worth stating directly, since this is where the real judgment sits. The rejected alternative was running every comment through the careful, slower pipeline all the time, no fast lane at all. It lost because adoption depends on trust built in the first few weeks, and an engineer waiting minutes for a naming nit uninstalls the tool before the slow lane ever gets the chance to prove its worth. The AI specific failure worth naming by name is silent under checking: a model that runs out of time produces no comment at all, rather than saying it didn't finish, so its silence is indistinguishable from an actual clean pass. The guardrail is forcing an explicit "ran out of time, not fully checked" comment the moment a trace gets cut off by its own clock, instead of letting it drop silently, so quiet never again gets read as clean. And the trade being accepted on purpose is real: the slow lane only runs a careful check on work that gets tagged auth or data scope adjacent by a fast, near instant static pass first, which means that tagging step itself has to be trusted, and a change that slips past the tag entirely still only ever gets the fast clock.

The four, in one line each:
P: fast under ten seconds for what a person would catch anyway, minutes for what only the model sees.
I: engineers feel seconds, the company and its customers feel days.
C: the hidden and expensive error is the one that ships silent, optimize against that one.
K: muting on the fast lane means pull back, a flat quarter on the slow lane means shrink it.

Same four moves, a pharmacy counter instead of a pull request

Not every hidden and expensive error ships inside a code repository. Sometimes it's a prescription, caught the same way: quietly, and only once someone downstream gets hurt.

Redwing Pharmacy Group runs Scriptcheck, a tool that reads a new prescription against a patient's existing medications and drafts an interaction warning before the pharmacist fills it. Iset Haverstock runs quality on it.

P, position: keep Scriptcheck's warning under about three seconds for common, well documented interaction pairs, the ones a pharmacist would likely flag from memory anyway. Let it run longer, up to about a minute, on a rare combination that needs a deeper look across a patient's full history, since that's the pair a pharmacist is least likely to catch unaided.

I, impact: testing too long costs a pharmacist real minutes at a busy counter, a line of customers watching the screen think. Testing too little risks a rare interaction riding home in a bag, discovered only once a patient has a reaction days later.

C, cost asymmetry: an extra few seconds at the counter is cheap, and every customer in line notices it happening. A missed rare pair is hidden, since a quiet screen looks exactly like a clean one, and expensive in a way that shows up later: an emergency visit, a report, a pharmacist who has to explain why the system said nothing.

There is no box between "rare pair not traced" and "fills as normal." That gap is where a quiet screen gets read as a clean one.

K, kill criteria: if the deeper check on rare pairs keeps coming back clean for a full quarter running, that's a sign to shrink its budget. If a rare pair still slips through inside the current time limit, the limit is too short, not the idea.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to the line: fast for what a person would catch anyway, slow for what only the model is checking.
Cost: there's no budget this quarter to keep the slow lane running on every tagged pull request. Shrink how many diffs qualify for it, tighten the tag, but keep the clock length the same for whatever still qualifies. The size of the slow lane can shrink. Its patience per check shouldn't.
The model got better, for real: say the new model is measurably faster at the same accuracy. That's exactly when it's tempting to fold the two lanes back into one, and exactly when you shouldn't, since the split was never really about speed. It was about which mistakes you're willing to let ride.

Where people run it wrong.
They set one clock for the whole product because it's simpler to explain, and simple isn't the same as safe.
They let a comment go silent when it runs out of time, instead of saying so, so a rushed check and a clean one look identical.
They forget to revisit the slow lane's budget once the model gets faster, so a limit set for last year's model quietly outlives its reason for existing.

How to use it live. Say the split out loud before naming a number: what's cheap to miss, and what only this tool is checking for. That buys a beat of thinking time, and it turns the rest of the answer into naming the actual line instead of guessing one number for everything.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0