When does building become a hiring decision rather than an engineering decision?

Fernbridge Support builds customer-support software. TicketSense drafts a reply to an incoming support ticket for an agent to review and send. Callum Streeter is the AI PM who owns it, and Reya Kessler is the senior engineer whose week quietly filled up with work nobody had ever assigned to her.

How to answer this, stage by stage

Nobody is scoring whether you can name a headcount number. They're scoring whether you'd notice the shift before it shows up as a missed roadmap item.

Let's learn

TicketSense reads an incoming support ticket and drafts a reply, which a support agent reviews, edits if needed, and sends, instead of writing the reply entirely from scratch.

None of these were ever written into anyone's job description. All four quietly became Reya's.

When TicketSense first shipped, Reya spent about 8 percent of her week on this kind of upkeep, small, occasional, and easy to absorb alongside her normal feature work.

By month three, it had crept to 19 percent, still under most people's radar, since Reya kept shipping her normal feature work alongside it, just at a slower pace nobody had connected to a specific cause yet.

Fernbridge had been living in the fourth branch for months without ever officially choosing it.

Here's the turn: the real cost was never the hours themselves. It was that TicketSense's roadmap, three planned features, quietly slipped a full sprint, and leadership initially read that as Reya being slower, not as a structural gap in who owned ongoing AI quality.

What I would leave alone: Fernbridge's internal expense-approval tool, a small rules-based helper with no model in it at all, needs none of this thinking. There's no drift to watch, so there's no ongoing AI-upkeep job to staff for.

The lesson: a hiring gap doesn't usually announce itself as a hiring gap. It shows up first as a senior engineer's calendar quietly filling with work nobody assigned, and only later as a missed roadmap item everyone notices and misdiagnoses.

Now here is the same thing as a story

The short version above is what you'd say out loud in the room. Read this one for what it actually felt like watching a real job form with no name and no owner.

Callum Streeter had shipped TicketSense to genuine relief from the support team: average reply time down, agents happier, a clean launch by most measures. Reya Kessler, the senior engineer who'd built most of it, kept a loose eye on it afterward, the way anyone keeps an eye on something they built.

Nothing about this looked like a crisis at any single point. It only reads as one laid end to end.

A model version update in month two changed how TicketSense phrased refund explanations, subtly enough that nobody flagged it for a week, until an agent noticed replies sounding oddly clipped. Reya fixed the prompt. Nobody assigned her to; she just did, because she understood the system best.

By month four, this had become a pattern: check after each update, patch the eval set when a new ticket type appeared, quietly handle the "why did it say that" questions that came her way instead of a support agent's. None of it was written down as her job. All of it was, in practice.

The fix was never asking Reya to work harder. It was giving the second job a name, and someone whose actual job it was.

The moment that surfaced it: a new engineer, in her second week, asked in a team meeting who was responsible for checking whether TicketSense's replies were still accurate after a model update. There wasn't a clean answer. Reya said, half-joking, "I guess that's me," and the room went quiet in a way that made it obvious nobody had actually decided that.

Callum never had a fixed rule for exactly when "someone doing extra AI upkeep" became "a role the team is missing." It came down to a feeling with two settings: either the upkeep was occasional enough that anyone could absorb it without noticing, or it had become a real, recurring share of someone's week that was quietly costing the roadmap elsewhere. Thirty four percent, discovered only because a new hire asked a plain question, was unmistakably the second setting.

Once named, the job turned out to have four real parts, not one vague catch-all.

Back when TicketSense first launched, letting Reya informally own its ongoing quality wasn't an unreasonable call, the upkeep really was light at first, and there was no reason yet to think it would grow. It stopped being reasonable the moment model updates and ticket volume both climbed and nobody ever separated "built the feature" from "keeps it accurate" as two distinct, differently-staffed jobs.

The time cost climbed for five months before the ownership axis ever moved to match it.

Here's the replay: Fernbridge opened a dedicated applied-AI engineer role that quarter, with eval pipeline upkeep, drift monitoring, and version migration explicitly named as the job, not implied. Reya's AI-upkeep share dropped back to a manageable 10 percent within two months, spent mostly reviewing rather than doing. The next model update landed without eating into anyone's roadmap at all.

One version of this story lets a senior engineer's week keep quietly filling until she burns out or the roadmap slips badly enough to force an uncomfortable postmortem. The other tracks the actual number, names the threshold in advance, and turns an invisible second job into a real, protected first one.

What I'd tell myself, hearing that new hire's question land in the room: the org had already been paying for this role in lost velocity for months. The only thing missing was a name for it, and a number that would have made the case months earlier.

LEAD, run on a job that existed before anyone named itNot a script for hiring the moment anyone looks busy. LEAD is what tells you a role is missing before the roadmap slips prove it.

The recap, one line per letter: link is protecting reliable AI quality without burning out one engineer, early signal is the share of a senior engineer's week on AI upkeep, tracked before the roadmap notices, abuse is the risk of that number falling because quality work quietly stopped instead of got resolved, and decision is the concrete threshold, roughly 30 percent for a full quarter, that actually triggers a real hire.

And if you want to be sure it really works, try it somewhere elseSame four letters, a legal-document startup instead of a support-software company. This time the early signal shows up in a different kind of hour.

Idris Fontaine is the AI PM at Caswell Legal Docs, which specifies ClauseCheck, a feature that flags risky clauses in a contract draft for a paralegal to review. Mapped onto LEAD: link is a flagging tool lawyers actually trust, not one they've learned to double-check entirely by hand. Early signal here isn't a senior engineer's calendar, it's the share of flagged clauses a reviewing paralegal overrides, climbing from 6 percent to 28 percent over four months as contract types diversified beyond what ClauseCheck was originally tuned on. Abuse is that the override rate could look artificially low if paralegals simply stopped bothering to override and started rubber-stamping the tool's flags instead, which would hide a real accuracy problem behind a falling number. Decision is similar in shape but different in trigger: past a 25 percent override rate sustained for a full quarter, Caswell opens a dedicated legal-AI engineer role, since a generalist engineer patching individual clause types one at a time can no longer keep pace with how fast contract variety is growing.

Same threshold logic, a different leading number. The shape of the decision travels even when the metric itself changes.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Skip straight to "track the share of a senior engineer's week on AI upkeep, hire once it holds above thirty percent for a full quarter," and stop.
Cost: no budget to open a new headcount line yet. Say so honestly, and commit to formally protecting part of one engineer's role as the interim step, rather than letting the informal arrangement continue unnamed.
The model got better, for real: say a new model version needs almost no prompt adjustment at all. The upkeep share would drop on its own, which is real good news here, not a sign the metric is being gamed, since nothing about the underlying eval or monitoring work stopped happening.

Where people run it wrong.
They wait for a missed roadmap deadline to notice the staffing gap, instead of tracking the leading number that predicted it months earlier.
They read a rising workload as one person needing to manage their time better, instead of as a role the organization hasn't hired for yet.
They hire for "AI support" with no named responsibilities, which just recreates the same undefined second job under a new title.

How to use it live. The moment an interviewer asks when building becomes a hiring decision, don't reach for a headcount number. Ask yourself: what's the one number that would show this happening months before a roadmap slips? That's usually the real answer hiding in the question.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0