How much model-level depth does an AI PM need before they become a bottleneck to their team?

Driftlock is Sabrehold Robotics' system for running a fleet of 40 picking robots through a warehouse: which shelf a robot drives to next, what path it takes, and whether it should attempt a grab itself or hand the item to a person. Ndidi Ozoemena owns the routing stack. Yorick Thrasher, the AI PM, spent two years around a robotics research lab before this job, enough to sound fluent about path planning, not enough to be reliably right about it. This is where that fluency stopped being useful and started costing the team something real.

How to answer this, stage by stage

Nobody is grading whether you can list good PM habits. They're grading whether you can point at the exact line between a call that's genuinely yours and one that only sounds like it should be.

Let's learn

Driftlock is Sabrehold Robotics' system for running a fleet of 40 picking robots through a warehouse. It decides which shelf a robot drives to next, what path it takes through the aisles, and whether it should try to grab an item itself or hand it to a person standing nearby. Before a near miss changed things, Ndidi Ozoemena tested a routing change against last week's aisle traffic in the simulator, watched it clear cleanly, and shipped it. Twice a month, sometimes three. Start to finish, about two days, with a short message to the team first.

Now the same kind of change waits. A routing or replanning tweak that used to take two days sits in Yorick Thrasher's queue for closer to two weeks, because he wants to personally weigh in before it ships.

Here's the turn. The wait itself was never the real problem. Eleven days is annoying, not dangerous. The real problem is what nobody was doing during those eleven days: watching the one number that actually needed Yorick's attention.

Six weeks before the near miss, Sabrehold added a new line of soft, floppy mailers to the pick catalog, items with no fixed shape for a robot's camera to lock onto. Nobody moved the 0.72 cut-off to match. Pick success on that one category fell from 94 percent to 81, meaning more items dropped and more damaged returns, roughly one pick in eight instead of one in seventeen. Across the whole fleet, the blended number barely moved, because the new category is only about one pick in nine. Leadership's weekly dashboard stayed calm the entire time.

Raise that cut-off and Driftlock catches more shaky grabs before they turn into a dropped item. It also sends more picks to a person, which costs labor time and slows the line. That trade belongs to Yorick, on purpose. It's the actual job. Deciding whether a routing algorithm replans every 200 milliseconds or 80 was never it.

What I would leave alone: routine tuning changes Ndidi tests only against historical aisle traffic, with no product-facing number attached, don't need Yorick's eyes at all. Not every technical decision is a threshold in disguise.

The lesson: a PM who understands a model well enough to ask good questions is not the same as a PM who's positioned to answer them. The fix for that gap is never "get better at the technical side." It's a written line saying which calls are actually yours.

Two dials on the same product. Only one of them was ever his to turn.

Now here is the same thing as a story

Read the walkthrough above when the clock is running. Read this one when you want to feel what six quiet weeks actually cost.

Ndidi Ozoemena has spent five years on Driftlock's routing stack, the part that keeps forty robots from driving into each other at two in the morning. She's good at it in a specific way: show her a near miss and she can usually name the exact number in the planner that needs to move, within the hour.

For most of her time at Sabrehold, that's exactly what she did. Test a change against last week's aisle traffic, watch it clear a few thousand simulated routes clean, ship it. Twice a month, sometimes three, always with a two-line message to the team first: here's what changed, here's why. Nobody signed off. It was her stack.

Nobody told her to stop asking on her own. She just learned, one message at a time, that it was safer to ask first.

Yorick Thrasher joined as AI PM eighteen months ago, straight out of a robotics-lab research post where he'd spent two years around exactly this kind of system, close enough to know the words, not close enough to have shipped one. For a while, that made him good at his actual job: he wrote the grasp-confidence threshold spec, built the eval set it gets checked against, ran the launch review. Ndidi liked working with him. He asked sharp questions and then got out of the way.

That started to change around the time the fleet grew from twenty robots to forty. More robots meant more tiny near misses, a few inches, nothing anyone reported. Yorick started reading the incident logs himself. Then he started asking Ndidi to walk him through her routing changes before she shipped them, not after. Not officially. Just a message: "hey, can we talk through this one first?" Once a month became once a week.

At 1:40 one morning, robot 17 and robot 24 both went for the same gap in aisle 12 and stopped four inches apart. Camera caught it. Nobody was hurt. Nobody even noticed until the morning review.

Ndidi knew the fix inside an hour: drop the collision-avoidance replanning interval from 200 milliseconds to 80, so a robot reacts to its neighbor faster. She'd made changes like it a dozen times. She wrote it up and sent it to Yorick, same as always, expecting a quick reply.

He wanted to review it himself first. Specifically, he wanted to understand the trade-off between a faster replanning cycle and the extra load it put on each robot's onboard processor, a real-time systems question with a real answer, just not one he was positioned to check. He asked her to re-explain it three different ways over the next week and a half.

The path only runs one way. Nothing in it lets Ndidi say a call is hers and have that be the end of it.

Ndidi had nowhere to take that. She could keep re-explaining, or she could ship it anyway and risk being the engineer who went around her PM on a safety fix, which is a much worse conversation than a slow one. She kept re-explaining. For two weeks after, she stopped proposing anything at all, not because there was nothing left to fix, but because asking felt safer than being asked to re-explain a fourth time.

In the retro afterward, someone floated making it official: a standing weekly meeting where every routing change gets walked through out loud before it ships. For about a day that sounded like progress. Then someone did the math. Forty minutes a week, forever, and it doesn't hand the call back to whoever actually owns it. It just builds a meeting around the same wait.

While he was reviewing her replanning interval for the third time, nobody was watching the grasp-confidence threshold, the number he'd written the spec for himself, eighteen months earlier. It hadn't moved since the soft-mailer category shipped six weeks before. Pick success on that category sat at 81 percent, quietly, under a fleet-wide average that barely blinked.

Here's the replay. Same near miss, one change made in advance: Ndidi owns replanning cadence outright, ships within two days like she always did, no sign-off required. Yorick spends that freed-up week doing his actual job, checking whether the grasp threshold still fits the current item mix. He catches the soft-mailer gap in week two instead of week eight, moves the cut-off, and pick success on that category climbs back toward 90 before it ever reaches a customer's return box.

One design hands Ndidi a real lane and Yorick a real dial. The other hands them a shared inbox and calls it collaboration.

What I'd tell myself, back when Sabrehold was four people and nobody thought to write any of this down: the day you hire someone whose job is to understand the model without building it, write down what that understanding is actually for. Otherwise it goes looking for something to decide.

GUARD, or the difference between understanding a dial and owning it

Not a checklist for spotting risk in general. GUARD forces you to say, specifically, who's stuck absorbing a technical opinion that was never supposed to be a technical decision.

Yorick has one lever, and he reaches for it on every call. Ndidi has the fix, and no lever of her own.

The whole Reduce step in one picture: one call, two places it can go, decided by what kind of call it is, not by who's in the room.

And if you want to be sure it really works, try it somewhere else

Same five letters, an insurance company instead of a warehouse, and the dial nobody's turning is a fraud-score cut-off instead of a grasp threshold.

Larke is a fraud-risk scoring tool an insurance company uses to triage incoming claims: it reads each claim and returns a score, low enough to auto-pay, high enough to route to a human investigator. Kasimira Owusua built and owns the feature set feeding that score, things like claim-to-policy-age ratio and prior claim density in the same zip code. Balthasar Chetwynd, the AI PM, keeps sending her feature proposals back with his own suggested replacements, engineered guesses pulled from a weekend of reading about fraud detection. Meanwhile the number that's actually his to own, the score cut-off that sends a claim to manual review, hasn't moved in eight months, even after claim volume in one region tripled and pushed the investigator queue past what the team could clear in a week.

Same two dials as Sabrehold's story. A near miss became a queue of unreviewed claims instead.

Same rank, different lever, mapped straight onto GUARD: the groups are Kasimira, whose feature proposals keep getting relitigated, and the claims team waiting behind her. The harm concentrates on Kasimira specifically, whose actual expertise, which signals predict fraud, keeps getting second-guessed by someone with less of it. She has no clean way to push back without her disagreement reading as difficult, since Balthasar's confidence sounds informed even when it isn't grounded. The fix is the same shape: Balthasar owns the cut-off and the labeled eval sample it's checked against, in writing, and defers feature selection to Kasimira. Detecting it means watching the investigator queue's wait time and how often Kasimira ships a feature without asking first, the same leading signal that would have caught Sabrehold's gap early.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip to it: write down two ledgers, thresholds are the PM's, algorithm and feature choices are the engineer's, then watch for decisions that start routing through the PM that never used to.
Cost: no time to formalize a written boundary this quarter. Start with the cheapest version, one paragraph on a shared page naming who owns what, then revisit it once there's room.
The model got better, for real: say Driftlock's grasp-confidence model gets meaningfully more accurate overall. The boundary still matters, because a better model just moves where the cut-off should sit, it doesn't remove the need for someone to own moving it.

Where people run it wrong.
They mistake technical vocabulary for technical judgment, and let a PM who can name an algorithm assume they can pick one.
They try to fix it by asking the PM to read more, instead of writing down who owns which call.
They watch overall throughput and miss that the real damage, a stale threshold, a growing queue, is happening on a dial nobody's tracking.

How to use it live. Ask the ownership question before naming a fix: "when a technical call comes up, is there a written answer for whose call it is, or does it default to whoever's in the room with the most confidence?" That question alone tells you whether the bottleneck is a person or a missing boundary.

Three things worth stating directly, since the real judgment sits here. The alternative Sabrehold could have taken instead of a written ownership boundary was giving Yorick formal veto power over every routing change, with a standing weekly review meeting to walk him through the reasoning. That was rejected in the retro: it would have made the bottleneck official instead of removing it, turning an eleven-day wait into a permanent one. The AI-specific failure worth naming is silent threshold drift under distribution shift: Driftlock's grasp-confidence cut-off was tuned against the item shapes in the warehouse at launch, and nothing forces it to get rechecked when a genuinely new shape of item, like a soft, compressible mailer, starts flowing through the same pick line. The guardrail is scheduled recalibration against a fresh golden set of picks every time a new item category ships, not just once at launch. And the trade-off is real: raising that cut-off catches more low-confidence grabs before they become a dropped or damaged item, but it sends more picks to a person, which costs labor hours and slows the line down. Setting that trade-off, deliberately, was always Yorick's job. Refereeing Ndidi's replanning interval never was.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0