How do you prevent a spike from silently becoming the production implementation?

Thornbury Field Services runs DispatchWrench, which listens to a technician's voice note about a job and suggests which parts to load before the truck leaves. Emeka Obidiegwu is the AI PM who has to explain how a two-week experiment on one engineer's laptop ended up deciding what forty of the company's trucks carry every morning.

How to answer this, stage by stage

Nobody is scoring whether you can spot a spike that obviously became production. They're scoring whether you can catch the one that never had a moment where anyone would've noticed.

Let's learn

Say we build a tool that listens to a technician's voice note describing a job and suggests which parts to load onto the truck before they even leave the shop.

Before DispatchWrench, a technician guessed which parts to bring based on a customer's phone description, and across 60 technicians, missed guesses caused an average of 1.8 return trips a week per technician, about 45 minutes each, roughly 81 hours a week company-wide.

The five letters, held up as one page. Identify the flip is the step this question is really testing.

Astrid Vance, an engineer at Thornbury, built a rough version in two weeks: her own laptop, a cron job, a hardcoded list matching keywords in a transcript to likely parts. In a 15-technician pilot, return trips dropped 70 percent, from 1.8 a week to about 0.54.

Here's the turn: the extra wrong suggestions on a new furnace model were never really the problem. The problem was that as more technicians relied on it, the one safety net, a supervisor's daily spot-check, quietly stopped, and nobody replaced it with anything, because DispatchWrench kept being right often enough that stopping felt harmless.

At its worst, DispatchWrench keeps recommending a discontinued part for a growing share of real jobs, technicians waste truck space and burn after-hours ordering fixing it themselves, and the one engineer who understands the script goes on vacation while it silently keeps running unmonitored on her personal laptop.

What I would leave alone: a small side-script three technicians use privately to sanity-check their own notes doesn't need any of this. The gate exists for tools other people start depending on without knowing it.

The lesson: a spike doesn't become risky the day it launches. It becomes risky the day people quietly stop treating it like something that could still be wrong.

Now here is the same thing as a story

The short version above is what you'd say proposing the graduation gate to leadership. Read this one for what four months of nobody noticing actually looked like.

Astrid Vance has a reputation at Thornbury for shipping something useful fast and then moving on to the next problem. DispatchWrench was supposed to be a two-week curiosity.

The pilot went well from day one. Fifteen volunteer technicians loved not guessing anymore, and word spread on its own. Nobody asked Astrid to expand it; techs just started asking dispatch to be added to the list.

Every arrow here happened on its own. No box in this row was ever a scheduled decision.

For the first six weeks, a shift supervisor spot-checked DispatchWrench's suggestions against real parts invoices each morning, a habit that started when the pilot was small enough to eyeball easily.

By month three, 41 of 60 technicians were relying on it daily, and dispatch had quietly started using its suggestions to pre-load trucks each morning before a shift even began. The supervisor's spot-check had stopped around week six, not from any decision, just from the fact that nothing had gone wrong yet.

One thing faded on its own. The other held steady, then failed on exactly the case nobody was watching for.

A new furnace model arrived from the supplier, and DispatchWrench, still running Astrid's original hardcoded keyword list, kept suggesting the old, discontinued part for every job involving it. By month four, that furnace model made up 22 percent of all jobs, and DispatchWrench got the part wrong on 64 percent of them.

Same tool, same laptop. Nearly four times the reach, and zero times the review.

The real question was never whether Astrid's two-week script was good enough to become real infrastructure. It was whether anyone had ever agreed on the moment it stopped being a spike, and it turned out nobody had, because nothing forced that conversation to happen.

Four things a real launch would have required. None of them were ever added, because nothing ever asked for them.

When the pilot first took off, someone said, "let's just let people opt in if they want it, no need to make a big process out of a helpful little tool," and it sounded reasonable, since forcing a process onto something voluntary and small felt like overkill.

One full-page image to carry the whole answer: it was never a dial of gradually increasing trust. It was a switch, and nobody ever flipped it on purpose.

Rerun the same four months with a graduation gate in place, triggered automatically at 30 percent adoption: the hardening review happens in month two, the keyword list gets a versioned update process before the new furnace model ever ships, and DispatchWrench never gets the chance to be wrong on 64 percent of a real, growing job type.

What I'd tell myself, watching that adoption line climb with nobody at the wheel: a tool becoming load-bearing doesn't announce itself. It just quietly stops being something anyone is still checking.

FLIPS, the drift that never announced itselfNot a script for distrusting every successful pilot. FLIPS is what tells you exactly which habit quietly stopped while the tool kept working.

The recap, one line per letter: find the person is Emeka, accountable for the drift, locate the habit is the supervisor's daily spot-check, identify the flip is checking daily versus not checking at all, pinpoint the old decision is having no plan to replace the informal check as adoption grew, and show the replay is a graduation gate catching the gap in month two instead of month four.

And if you want to be sure it really works, try it somewhere elseSame five letters, a small-town newsroom instead of a truck fleet. The missing gate is still the same shape.

Odette Marchetti edits the Coldharrow Gazette, where BriefDraft, an intern's two-week experiment, drafts short recap articles from public meeting transcripts. Mapped onto FLIPS: find the person is Odette, who has to sign off on whatever the newsroom actually publishes. Locate the habit is a junior reporter fact-checking every BriefDraft output line by line against the transcript, for the first month. Identify the flip is a workaround flip, not over-trust: once the reporter trusted the drafts enough, she stopped opening the full transcript at all and instead only skimmed BriefDraft's output, quietly changing what "fact-checked" even meant at the paper. Pinpoint the old decision is never naming who owns BriefDraft once the intern who built it graduated and left. Show the replay names a permanent owner and restores a required transcript spot-check for one story per week, chosen at random.

A different building entirely, and the gap between reach and readiness looks exactly the same shape.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "set the graduation trigger before it succeeds, since success is exactly when nobody wants to stop and check," and stop.
Cost: no budget for a full hardening review yet. Say so honestly, and put a hard usage cap on the spike instead, one that has to be deliberately raised, rather than let it grow unchecked for free.
The spike really did stay small, for real: if a tool genuinely never crosses into being load-bearing, that's a fine, permanent state, and saying so is what keeps the gate from becoming process for its own sake.

Where people run it wrong.
They treat a spike's early success as proof it no longer needs the informal check that helped prove it out.
They let usage grow with no stated threshold that would force a hardening conversation.
They assume someone would notice a spike quietly becoming load-bearing, instead of naming an owner who's actually watching for it.

How to use it live. The moment you're asked how to stop a spike from becoming production by accident, ask yourself: what number, if crossed, should force a review nobody gets to skip? Name that number before the spike starts, and the rest of the safeguard follows on its own.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0