What does an AI infrastructure PM own that an applied AI PM does not?

Priorlens is Emberlock AI's shared engine for prior-art search. It turns a patent claim, or any block of prior art text, into a vector, then ranks every other document in its index by how close it sits in that same space. Osborne Duffield owns that shared engine. Kiernan Penrose owns ClaimSentry, the tool patent attorneys run at drafting time to check a claim's novelty before filing. Theodric Ndiaye owns FilingWatch, which watches competitors' new filings around the clock for infringement risk. Cadmore Sylvane, Emberlock's VP of product, asked Osborne this question the week both of their teams found out, on the same Wednesday, what happens when the engine underneath them changes and nobody says so.

How to answer this, stage by stage

Nobody is grading whether you can name two job titles. They are grading whether you can rank what actually belongs to each one, and defend the order, when a VP wants to know why two teams broke on the same day.

Let's learn

What happens when two products quietly share the same engine, and nobody tells either team when the engine changes? Priorlens is Emberlock AI's shared engine for prior-art search. Feed it a block of patent claim text, or a competitor's newly published filing, and it turns that text into a vector, then ranks every other document in its index by how close it sits in that same space.

Before ClaimSentry existed, a patent attorney checking one claim's novelty spent about six hours searching prior art by hand across three or four databases before feeling sure enough to file. With ClaimSentry, that first pass drops to about four minutes: paste the draft claim in, get a ranked list of the closest prior art, plus a flag if anything sits close enough to threaten novelty.

Osborne built the retrieval logic inside ClaimSentry's own codebase at first. It only became a real, separate job the day a second team needed the exact same thing for a different reason.

Eighteen months after ClaimSentry launched, Emberlock shipped FilingWatch on that same engine: instead of one draft claim, it reads every newly published filing from named competitors, continuously, and alerts Theodric's customers the moment something looks close enough to their own portfolio to matter.

Here is the turn. The extra mistakes were never really the problem, because for months there weren't any. Applied science retrained the embedding model underneath the whole engine, and on their own eval set it got a real, honest ten and a half point better at reading software-class patents, the exact area where the old model was weakest. They shipped it. And two teams, calibrated to the old model, started acting on numbers that no longer meant what those numbers used to mean.

What it costs at its worst: a real competing patent slips past FilingWatch for 19 days, past the point where the cheap, early challenge is still open, so the customer firm now has to use the far more expensive route to contest it, roughly ten times the cost. At the same time, ClaimSentry floods attorneys with false novelty warnings on claims that were actually fine, so they start ignoring the flags altogether, right when a true one might show up.

Nobody decided, on any single day, to let this run for 19 days. It just kept not getting caught, until a paralegal happened to look somewhere nobody had assigned themselves to look.

What I would leave alone: applied science's own internal research tool, which also calls the engine to explore old filings for pattern-mining, doesn't need any of this. Nobody there is calibrating a hard threshold or shipping a customer-facing decision off its score, so a version change there really is a non-event.

The lesson: a gap that only gets closed after a missed filing is not really closed, it has just been paid for once. Owning the shared engine means catching the next quiet version change before two teams calibrate around it again, not writing a nicer postmortem about the last one.

Now here is the same thing as a story

The short version above is what you actually say in the room. Read this one for the eighteen months that put Osborne in front of Cadmore's question in the first place.

Every Tuesday morning, Osborne Duffield opens the same dashboard before most of the building has its coffee, not because anything is usually wrong, but because for eighteen months, nothing ever had been.

He built Priorlens's retrieval logic inside ClaimSentry's own codebase, back when Emberlock had exactly one product and Kiernan Penrose was the only person who cared how it behaved. That made sense. There was nothing to share yet. Osborne tuned the ranking, argued with Kiernan about thresholds over lunch, and shipped whatever change felt right that week, because the two of them were the whole audience.

Eighteen months in, Emberlock launched FilingWatch. Theodric Ndiaye needed the exact same engine for a completely different job: instead of one draft claim checked once, a continuous stream of new filings checked every hour. Osborne pulled the retrieval logic out of ClaimSentry's repo and stood it up as its own service, the Priorlens Retrieval Core, one endpoint both teams called. He meant to write down the rules for changing it later, once things settled.

Neither team trusted the other to notice a change, because nobody had ever been assigned to notice it centrally. So they each built the same watch, alone.

Things never settled. What Osborne didn't know, until much later, was that both teams had quietly built their own morning script: a fixed set of known claim and prior-art pairs, pinged against the engine, just to notice if the scores had moved. Kiernan's team built theirs first. Theodric's team, not knowing Kiernan's existed, built the same thing about two months later. Two weeks of engineering time, twice, to solve a problem neither of them had been asked to own and neither of them knew the other had already half-solved.

Applied science's retrain of the embedding model was, by every number they owned, a real and honest win: top-ten retrieval precision on software-class patents, historically the model's weakest area, climbed from 71 to 84 percent. They swapped it in on a Wednesday in March, behind the same endpoint, because it was "just a model update, same API," the kind of change that had never needed a heads-up before.

One change. Two teams, still reading the old rulebook for what a "close" score meant, broke in two different directions at once.

Nobody noticed for 19 days. Kiernan's team saw ClaimSentry's false-block rate creeping up and assumed attorneys were just filing messier claims that month. Theodric's team saw nothing at all, because FilingWatch's dashboard doesn't show what it failed to flag, only what it did.

Faolan Pierce, a paralegal at Ashvale & Greerson, one of FilingWatch's customer firms, found the gap by accident. Searching a rival's name for an unrelated reason, she came across a newly granted patent that looked, on a plain reading, close enough to her client's own claims to matter. FilingWatch had never flagged it. The filing had been public for 19 days, long enough that the firm's cheap, early challenge window had already closed. They would have to use the far more expensive route instead, roughly ten times the cost of the one they'd missed.

The decision Osborne would take back sits in a hallway conversation from eighteen months earlier, back when Priorlens still lived inside ClaimSentry's own repo. Someone asked, in passing, whether they should write down rules for how a future model change would roll out. "When we actually have two teams depending on it" was the answer, and at the time that was a completely reasonable thing to say, because they didn't. Nobody wrote a reminder for the day that stopped being true.

Run the same eighteen months again, with a real deprecation policy in place the day FilingWatch went live: any new model version gets its own versioned endpoint, the old one stays callable for a six-week migration window, and a shared eval harness runs the same golden claim and prior-art pairs against both versions every night, flagging anything that moves. The March retrain still ships, still gets its real 71-to-84 win. But the harness catches the score drift on the golden set the very next morning, not 19 days later. Kiernan and Theodric each get six weeks to recalibrate their own thresholds against v3, on their own schedule, while v2 keeps serving traffic underneath them. Faolan never has a reason to search that rival's name out of worry, because the filing that would have slipped through gets caught by FilingWatch on day one instead.

What Osborne would tell his past self, back in that hallway: a rule you write "when we actually need it" only gets written on the day something breaks, because by definition, that's the first day anyone remembers to ask again.

ORDER, for the day a second team starts calling the same thing

GUARD would fit if this were about one attorney with no way to push back on a wrong flag. It isn't. The real question is what to build first, and in what order, once two teams already depend on the same engine, which is ORDER's job.

One of these you can undo before lunch. The other one already cost a customer 19 days and a tenfold jump in what it takes to fight back.

The second box is the only one nothing feeds into on its own. It has to happen, on its own, before the rest of the chain is worth building.

Three things worth stating directly, since the real judgment sits here. The alternative worth naming and rejecting is letting each team pin and run its own private copy of the embedding model. It loses because it multiplies infra cost, still lets the two copies drift apart from each other over time, and hides the coordination problem instead of solving it, two private wrong answers instead of one shared right one. The AI-specific failure worth naming by name is embedding-space drift hiding behind a real accuracy win: retraining a model changes the shape of the space its scores live in, so a threshold calibrated against the old space can silently stop meaning what it used to mean, even while the model gets more accurate by every measure its own team is watching. The guardrail is the versioned endpoint plus the nightly shared eval harness, comparing the same fixed set of claim and prior-art pairs across both model versions and flagging any pair where the two disagree past a set margin. And the trade-off is real and accepted on purpose: running v2 and v3 live in parallel for a six-week migration window costs roughly 40 percent more in serving cost during that window, against a 19-day gap that already cost one customer a tenfold jump in what it takes to contest a missed filing.

And if you want to be sure it really works, try it somewhere else

Same five letters, a claims desk instead of a filing cabinet, and the thing that breaks in two directions this time is a fraud flag and a payout estimate instead of a novelty check and a competitor alert.

Coldwater Mutual built a shared Extraction Core: one model that reads scanned claims documents, including handwritten repair estimates, and turns them into structured fields and a confidence score. Astoria Bellisario owns that core. FraudPulse, an applied product, scores how likely a claim is staged. SeverityLine, a second applied product, estimates repair cost from the same extracted fields. Both call the same core, calibrated against the same confidence numbers.

Different desk, same shape of danger. The thing two teams share is exactly the thing worth a real contract. The tweaks each team owns alone are not.

Coldwater's applied science team retrained the extraction model to read handwritten repair estimates better, a real win, field-level accuracy on handwritten claims rose from 68 to 81 percent, and rolled it out behind the same endpoint with no separate version. FraudPulse's fraud-flag rate on legitimate claims, previously a steady 3 percent, climbed to 17 percent, because the model's new confidence scores read differently than the old ones FraudPulse had been calibrated against. SeverityLine, tuned the opposite way, started under-estimating repair cost on the same handwritten claims, delaying real payouts by an average of 11 days while adjusters manually corrected the estimate.

Same rank, mapped straight onto Coldwater: Astoria needed a versioned Extraction Core endpoint before anything else, since FraudPulse and SeverityLine were both reading the same score two different ways. A deprecation policy would have given both teams a six-week window to recalibrate before the new model became the only option. A shared eval harness, run against a fixed set of real handwritten claims, would have caught the field-accuracy shift before either team's numbers moved. Cost and usage tracking mattered least here, since neither team's volume was the problem, the shared confidence score was.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: own the contract and the deprecation policy first, because those are what break every downstream team at once if nobody owns them.
Cost: no budget this quarter for a shared eval harness. Ship a manual weekly diff of the golden set first, free, and automate it once the budget exists.
The model got better, for real: say the next retrain reaches 95 percent precision. Both applied teams still need a deprecation window, because being more accurate on average was never the same claim as being safe to swap in silently.

Where people run it wrong.
They treat a climbing eval number as proof the whole system got safer, without checking whether anyone downstream had calibrated against the old version.
They centralize everything the moment one team complains, instead of checking whether a real second team actually depends on the same thing yet.
They write the deprecation policy after the second incident, because the first one gets explained away as a one-off.

How to use it live. When an interviewer asks what infra owns that applied doesn't, ask yourself one thing before answering: what happens to every other team the day this one team's mistake gets fixed by someone just reverting it? If the answer is "nothing, they were never touched," it isn't infra. If the answer is "they would all break too," that's the thing infra owns.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0