CaseIntermediateModel Fluency & the AI PM Role / AI PM role variants: platform, applied, infra, research / #5

Which role variant would you assign to owning the internal prompt library, and why?

ORDER · ranking who owns Placecard's shared prompt library, at Tarnhollow Labs' restaurant reservation and no-show tool

Placecard is Tarnhollow Labs' reservation and no-show tool for independent restaurants. Four separate flows, reminders, waitlist offers, win-back texts, and confirmations, all draw their guest-facing wording from the same shared prompt library. Wysteria Deniau owns it today, by accident, because her team built it first for their own no-show-risk feature. Her VP wants to know which role variant should actually hold it, the week after a guest at The Wrenford no-showed a table Placecard itself had scored as risky.

The direct answer
Assign the shared prompt library to the platform PM, not to Wysteria's applied team just because they built it first. It is cross-team infrastructure: four separate guest-facing flows read from the same templates, so it needs an owner accountable to all four, with a version-controlled eval set checked every time the underlying model changes. Leave it with an applied PM and it gets optimized for her own feature's numbers, while every other team's prompts quietly drift, and by the time anyone notices, weeks of guest texts have already gone out wrong.
Do this, in order
  1. Assign the shared prompt library to the platform PM.Why: only platform is accountable to every flow that reads from it, not just the team that happened to build it first.
  2. Build one version-controlled eval set covering all four flows, before the next model swap, not after.Why: leaving this gap open is exactly what let a bad edit run six weeks unwatched.
  3. Require every flow that reads from the library to register itself as a consumer.Why: nobody can own what they can't see is being used.
  4. Reject applied-PM ownership, even though it's the path of least resistance.Why: an applied PM will always tune the library toward her own feature's numbers first.
  5. Reject infra-PM ownership.Why: infra keeps the pipe fast and up. It was never built to catch a sentence that quietly stopped saying what it used to.
  6. Reject research-PM ownership.Why: research is paid to look at the next model, not to keep today's shared asset stable for four teams already depending on it.

How to answer this, stage by stage

Nobody is grading whether you can name four job titles. They're grading whether you can rank them, and defend the top pick, when the wrong owner has already cost a real restaurant real money.

1
Ground it in one real library, not a debate about job titles
Say it like this
"Let's make this concrete. Placecard is Tarnhollow Labs' reservation and no-show tool. Four flows, reminders, waitlist offers, win-back texts, confirmations, all pull guest-facing wording from the same shared prompt library. Right now an applied PM named Wysteria owns it, by accident, because her team built it first."
Why this works
A real library with a real accidental owner keeps this from turning into an abstract debate about job titles.
2
Name your structure before ranking anything
Say it like this
"I'll run this as ORDER. Outcome, what owning it well actually protects. Reversibility, how bad the wrong owner turns out to be. Dependency, what has to be true before anyone can own it well. Evidence, what's cheap to check. Rank, the actual answer, defended."
Why this works
Two seconds of structure tells the interviewer you have a method, not just an opinion about four job titles.
3
Reframe what the question is really testing
Say it like this
"This isn't really 'who gets credit for building it.' It's 'who answers for it the day a model swap quietly changes what every team's prompts say, not just the team that happens to be watching.' Those are two different questions, and most candidates only answer the first one."
Why this works
This line is the whole answer in miniature. Skip it and the ranking sounds like a turf argument instead of a real judgment call.
4
Give the ranked answer, committed
Say it like this
"So here's my answer. The platform PM owns it. Not the applied PM whose team built it, not infra, not research. This is cross-team infrastructure, and only platform is accountable to every flow that depends on it, not just one."
Why this works
This is the direct answer, spoken, before a single number distracts from it.
5
Prove it with the real drift, numbers first
Say it like this
"Here's what happened with an applied PM holding it. Tarnhollow swapped to a cheaper model to cut inference cost. Wysteria's team retested her own reminder prompts and shipped. Nobody retested the waitlist and win-back prompts, because nobody owned them. Six weeks later, 15 percent of the 'all set' texts were going to reservations Placecard's own model had flagged elevated risk."
Why this works
A real number climbing the wrong direction beats any amount of talk about ownership in the abstract.
6
Name and reject each alternative before the interviewer does
Say it like this
"Three real alternatives, and here's why each loses. Applied PM optimizes for her own eval set, and everyone else's prompts rot silently, that's exactly what happened. Infra PM owns whether the pipe is fast and up, not whether the words are still calibrated, those are different jobs. Research PM owns exploring the next model, not stewarding a stable asset four teams already depend on today."
Why this works
Naming and rejecting the alternatives yourself beats waiting for a follow-up question to expose the gap.
7
Close on the one checkable line
Say it like this
"So: the platform PM owns the prompt library, because it's cross-team infrastructure, not one feature's side project. You'll know it's working when the next model swap gets caught by a shared test suite in two days, not by a guest calling to ask why nobody warned her."
Why this works
Ends on something checkable, not just a confident-sounding ranking.

Let's learn

Placecard reads a restaurant's own booking history, a guest's past no-show record, and how far out a reservation was made. It scores that reservation's no-show risk, then drafts the text the guest actually receives: a confirmation, a reminder, a waitlist offer, or a win-back message after a miss.

Hand sketched labeled parts diagram titled One library, four flows pulling from it. A central document icon labeled Shared prompt library, with four callouts radiating out: Reminder texts, Waitlist offers, Win-back texts, Confirmation texts.
One library. Four flows quietly depend on the exact same sentences, whether anyone planned that or not.
Knowledge spark: what makes a reservation "elevated risk"? Placecard's model weighs a guest's own no-show history, the party size, and how far out they booked. A first-time caller booking a Saturday table three weeks out for eight people scores higher than a regular who always shows, booked yesterday for two. Elevated risk doesn't mean the guest is doing anything wrong. It means the reminder text needs to actually ask them to confirm, not just say hello.

Before Placecard, The Wrenford's no-show rate ran about 18 percent, a number GM Josiah Marchetti had lived with for years. With Placecard's reminder texts, worded differently depending on how risky a reservation looked, that dropped to 9 percent within two months, including the elevated-risk tier, the reservations most likely to no-show in the first place.

Hand sketched comparison diagram titled Same guest, same risk score, two different texts. Left panel, a document icon labeled Before the migration, caption elevated risk got please confirm by 6pm. Right panel, a question mark icon labeled Six weeks after, caption same elevated risk got you're all set.
Same risk score, same guest, two different sentences. Only one of them asks the guest to do anything.

Here's the turn. Placecard's model never got worse at spotting risk. What broke was the one sentence that told the guest to do something about it.

Share of elevated-risk reservations texted "you're all set" language, by week after the model migration
20% 10% 0% 3% 5% 8% 10% 13% 15%, found here Wk 1 Wk 2 Wk 3 Wk 4 Wk 5 Wk 6
Elevated-risk reservations texted "all set" instead of "please confirm"
Nobody watched this number, because nobody owned all four flows at once. The team that owned the reminder flow only watched its own eval set, which never touched the waitlist or win-back prompts drawing from the same shared template.
We didn't lose the ability to spot a risky reservation. We lost the one sentence that told the guest to do something about it.

What it costs at its worst: across those six weeks, about 19 reservations that Placecard's own model had flagged elevated risk no-showed anyway, guests who had been told, in plain words, that they were all set. At The Wrenford's average of 3.4 guests a table and 71 dollars a head, that's roughly 4,600 dollars walking out the door over six weeks, for one restaurant. Placecard serves 340 restaurants. If even a third of them route their elevated-risk texts through the same shared template the same way, the same six weeks cost the platform an estimated 520,000 dollars, and the one dashboard leadership actually watched, blended no-show rate across every tier, moved from 10 percent to 11. Barely a ripple.

The choice I would take back When Wysteria's team first built the risk-explainer prompt, nobody assigned it a named, cross-team owner or a shared test process, because at the time only her own feature used it. That was a reasonable call for a library one team used. It stopped being reasonable the day the waitlist team forked her template with no formal handoff, and it kept not being fixed for over a year after a second team did the same thing.

What I would leave alone: the small number of high-risk reservations that trigger an actual phone call from Wrenford staff instead of an automated text. A person writes and reads that message, so the shared library never touches it, and it never drifted. Nothing about this fix needs to go anywhere near it.

The lesson: a shared thing with no named owner isn't ownerless. It's quietly owned by whoever built it first, for exactly as long as it takes their private idea of "good" to become everyone else's default, without anyone deciding that on purpose.

Now here is the same thing as a story

The short version above is what you actually say in the room. Read this one for the year and a half it actually took to notice who was holding the keys.

Wysteria Deniau is good at her job. She built Placecard's no-show-risk score from a rough first pass into the feature that made Tarnhollow Labs' pitch to independent restaurants land: not just a prediction, but a plain-language text that told a guest exactly what to do about it. She wrote that language herself, tested it against her own set of no-show scenarios, and it worked.

Other teams noticed. The waitlist team needed to tell a guest their table might open up. The win-back team needed to text someone who'd already no-showed once. Neither team had anyone who wrote guest-facing copy as well as Wysteria's reminder prompts did, so they copied her templates, changed a few words, and shipped. Nobody asked her. She didn't mind. It felt like a compliment.

Hand sketched horizontal timeline titled How the drift crept in. Four milestones: Library built, caption for one reminder feature. Three flows fork it, caption no owner, no versioning. Model migration, caption cheaper model, cost cut, emphasized. New hire's question, caption week 6, drift found.
Nobody at Tarnhollow Labs decided, on any single day, that one team's prompt library had quietly become everyone's shared infrastructure. It just happened, the way it usually does.

Josiah Marchetti built a habit of his own. For years at The Wrenford, he'd personally called every reservation that felt shaky, a first-timer, a big party, someone booking weeks out. Once Placecard's reminder texts started doing that work for him, reliably, he stopped. He had a dining room to run. The no-show rate held at 9 percent for a year. He trusted the number the way you trust a thing that has never once let you down.

Then Tarnhollow Labs swapped Placecard's guest-facing model for a cheaper one, to cut the per-text inference cost that scaled with every restaurant on the platform. A sensible move, on its own terms. Wysteria's team retuned and retested the reminder prompts against her own eval set of no-show scenarios, and shipped. She had no reason to test the waitlist or win-back prompts. She didn't own them. She didn't know, day to day, that they even still pulled from her library.

The shared instruction underneath all four flows told the model to "state the guest's reservation status in one plain, confident sentence." It never spelled out the three tiers by name, all set, please confirm, needs action, it left the model to infer which sentence sounded "confident" for a given risk score. The old model had settled on three distinct sentences. The new one, run through the exact same instruction, leaned toward reassurance by default, and started collapsing the middle tier, please confirm, into the top one, all set.

Nobody saw it happen. The blended no-show number Josiah watched barely moved, 10 percent instead of 9. Elevated-risk reservations are only about a fifth of his bookings, so a real problem there gets diluted almost out of sight in a number that averages every tier together.

Nobody typed the wrong sentence on purpose. Nobody read the drift either, because the one number everyone watched was built to hide exactly this kind of gap.

What actually surfaced it was a question, not a report. Yestin Okwuosa, three weeks into the waitlist team, was reading two of the shared templates side by side while onboarding, and asked his lead why the "holding your table" version and the "confirmed" version said basically the same thing for two different risk scores. Nobody on his team could answer him cleanly, because nobody owned the whole library, only their own corner of it.

The decision Perceval Quenneville, Tarnhollow's platform PM, would take back sits in a meeting that never happened. When the waitlist team first forked Wysteria's template, over a year before the migration, nobody flagged that a second team touching the same shared instruction meant it needed a real owner and a real test process. It felt too small to escalate. One more team using a good template seemed like a win, not a governance gap.

Run the same six weeks again, with Perceval already holding the library and a version-controlled eval set covering all four flows. The migration still ships, since it's a real cost win. But the test suite runs the exact same risk-tier prompts through the new model before anyone else does, and the elevated-risk tier's sentence comes back wrong on day one, not week six. The fix ships before a single guest text goes out worded incorrectly. Two days, not forty-two.

What I'd tell myself, looking back at that first fork: a shared thing being useful to more people was never the moment it needed an owner. It was the moment two people who didn't talk to each other started depending on the same words for different reasons.

ORDER, for handing someone the keys instead of leaving them with whoever grabbed them first

FLIPS would fit if this were only about Josiah's habit of trusting the reminder texts. But the actual question asks for a ranked answer among four fixed roles, which is ORDER's job.

OOutcome. What owning the library well actually protects.
Every one of the four role variants could plausibly claim some piece of this. What owning it well actually protects is that every team's guest-facing text stays reliable and calibrated as the underlying model changes, not just the reminder flow Wysteria happens to watch. A rising accuracy score in the no-show model means nothing if the sentence built on top of it quietly stops matching what the model just decided.
Name the outcome before ranking anyone. Skip this step and a ranking is just four opinions in a row.
Hand sketched left to right flow diagram titled What has to be true before anyone can own this well. Three connected boxes reading Multiple teams, this box in dark ink, Real tests, this box emphasized in amber, Clear owner, this box in dark ink.
The middle box was the actual gap at Tarnhollow Labs. Four teams, zero shared tests, for over a year.
RReversibility. How bad the wrong owner turns out to be.
If an applied PM owns it, she optimizes the library toward her own feature's eval set, because that's the number she's measured on. Every other team's prompts atrophy quietly, since nothing forces her to check them. This mistake compounds: the longer it runs, the more teams have already forked or copied the drifted base template into their own flows, and untangling whose version came from where gets harder every month it goes uncorrected. That's exactly what happened here, over roughly eighteen months.
This is the step that makes applied-PM ownership the wrong default, not just a suboptimal one. The damage doesn't undo itself when someone finally notices.
Hand sketched comparison diagram titled Reversible, or not. Left panel, a gauge icon labeled Caught with tests, caption a bad edit across all four flows ships a fix in 2 days. Right panel, a scale icon labeled Left to one team's eval set, caption everyone else's prompts quietly rot for weeks.
One of these you catch on a Tuesday. The other one you find out about from a new hire's question, over a year later.
DDependency. What has to be true before any variant can own it well.
Two things, and only one of them existed at Tarnhollow Labs. First, more than one team actually has to be using shared prompts, which was true from the moment the waitlist team forked Wysteria's template. Second, a real versioning and testing process has to exist, which never did. Assigning a platform PM to this library without also building that test process is ownership in name only, a title on an org chart with nothing underneath it.
This is why the rank isn't just "give it to platform and walk away." Half the fix is the eval set, not the org chart line.
EEvidence. What's cheap to check, to know the current ownership is failing.
One cheap check: pull the last three edits to any shared template and ask who tested them, and against which flows. If the honest answer is "just my own eval set," that's the failure sitting in plain sight. A second cheap check: ask whether any two flows currently share a base instruction with no documented owner. At Tarnhollow, that answer was yes, for three separate flows, for over a year, and nobody had ever asked the question out loud.
Cheap to run, and it's exactly the kind of check that would have surfaced this gap the week the waitlist team forked the template, not eighteen months later.
Days to catch a broken shared prompt, before and after platform ownership
50 days 25 days 0 42 days Applied-PM owned no shared eval set 2 days Platform-PM owned tests all four flows
Same size of mistake, a shared instruction resolved differently by a new model. The only thing that changed between the two bars is who was accountable, and whether a test existed to catch it.
Hand sketched quadrant diagram titled Four role variants, one library. X axis, how close to one feature's day job, from far to close. Y axis, how many teams depend on it right now, from few to many. Applied PM sits close and moderately many. Infra PM sits mid close and few. Research PM sits far and few. Platform PM, emphasized, sits far on the x axis and many on the y axis.
Platform PM is the only one of the four who sits far from any single feature's day job and close to every team's dependency at once. That's the actual shape of the job, not a preference.
RRank. The four role variants, in order, defended.
Platform PM, first, because the outcome is cross-team reliability and only platform is structurally accountable to every flow, not one. Applied PM, second, and only as a contributor who flags drift in her own flow, never as the owner, because her incentives point at her own feature's numbers. Infra PM, third, because keeping the model-calling pipeline fast and up is a real, separate job that doesn't touch whether the words themselves are still calibrated. Research PM, last, because exploring the next model is forward-looking work, and stewarding a stable asset four teams depend on today needs someone who isn't already looking past it.
If this ranking would stay identical with a different outcome in the O step, say "platform PM always wins," it was ranked by habit, not judgment. Change the outcome to "ship the newest model as fast as possible" and research PM's rank would move. It doesn't move here, which is how you know this rank is real.
The check that keeps this ranking honest Swap the outcome and watch whether the order moves. If the only thing that mattered were shipping model upgrades fast, research PM might rank higher. It doesn't rank higher here because the outcome that actually matters is a hotel-sized restaurant chain's worth of guest texts staying true to what the model just decided, night after night, not how quickly the newest model gets tried.

Three things worth stating directly, since the real judgment sits here. The alternative worth naming and rejecting isn't only the other three roles, it's also "don't centralize it at all, let each flow fork its own copy of the templates." That option loses because forks don't stay in sync, they drift further apart with every edit, which is a slower version of exactly the failure that already happened once. The AI-specific failure worth naming is a shared instruction that leans on the model's own judgment of tone instead of an explicit, enumerated contract: "state the status in one confident sentence" let two different models resolve the same risk score into two different sentences, and nothing caught it because nothing was checking. The guardrail is the version-controlled eval set itself, one exact expected sentence per risk tier per flow, run against any candidate model before it ships to guests. And the trade-off is real and accepted on purpose: running that full four-flow suite adds roughly half an hour to every shared prompt edit, and it delayed the next cost-saving migration by two extra days while the team fixed what the tests caught. Slower shipping, on purpose, for a shared asset four teams already depend on.

And if you want to be sure it really works, try it somewhere else

Same five letters, a city permit office instead of a dining room, and the thing nobody separates this time is a routine reminder from a violation notice that actually matters.

Groundwork, built by Wickfield Civic Systems, reads a city's inspection schedule and a property's past compliance record, predicts which scheduled inspections are likely no-shows, and drafts the text a property owner receives. Perpetua Iwu, the applied PM for the inspection-reminder feature, built Groundwork's shared notice library the same way Wysteria built Placecard's, for her own flow first, copied informally by the violation-notice team afterward.

Hand sketched icon list titled Same shape, a city permit office instead of a dining room. Five rows: Groundwork predicts inspection no-shows and drafts the notice text. One applied team built the shared notice library alone. A phrasing update was retuned for reminders only. Violation notices quietly started reading like routine reminders. Platform PM took ownership, same rank, same reasons.
Different office, same missing owner. The library doesn't know or care which city it's serving.

When Wickfield Civic Systems updated Groundwork's phrasing rules for a new inspection code year, Perpetua's team retested the reminder flow only. The violation-notice flow, forked from the same base instruction, started reading exactly like a routine reminder for property owner Boniface Ekwueme, whose actual violation notice said "your inspection is coming up" instead of naming the code violation itself. He missed the real deadline, because the text never told him it was different from any other appointment reminder.

Same rank, mapped straight onto Groundwork: Perpetua had to be the one who first noticed the gap, since she owned the flow that leaked it, but Groundwork's platform PM is who should have owned the shared library from the start, for the same structural reason as Placecard: violation notices, reminders, and appeal-deadline texts all pull from one base instruction, and only a cross-team owner is accountable to all three at once.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: platform PM owns any shared prompt library, because it's cross-team infrastructure, and applied ownership optimizes for one team's numbers while the rest quietly drift.
Cost: no budget this quarter for a dedicated platform hire. Have the platform PM who already owns the model-serving roadmap absorb the library and its eval set as one line item, rather than leaving it further split.
The model got better, for real: say the new model is measurably more accurate across the board. Keep the shared eval set anyway. A better model can still resolve an underspecified instruction differently than the old one did, and better was never the same claim as calibrated.

Where people run it wrong.
They give ownership to whoever built the thing first, mistaking authorship for accountability.
They assign a title without building the version-controlled test process underneath it, so ownership exists on the org chart and nowhere else.
They watch one blended metric that dilutes exactly the slice where the real damage is happening.

How to use it live. Before ranking anyone, ask yourself out loud: "if this drifts, who is the one person every affected team can go to?" Whichever role variant answers that question honestly is usually the exact rank an interviewer is listening for.

Flashcards (tap any card to flip it)

1 · THE FRAMEWORK
Which framework fits "which role variant would you assign to owning the internal prompt library, and why?"
Tap to flip
ANSWER
ORDER: outcome, reversibility, dependency, evidence, rank. Built for ranking a fixed set of candidates by what breaks first if the wrong one gets picked, which is exactly this question's shape.
2 · THE CAST
Who are the four role variants, and who is this story about?
Tap to flip
ANSWER
Applied PM Wysteria Deniau, Platform PM Perceval Quenneville, Infra PM Amberlyn Vantager, Research PM Amadeo Correlli, all at Tarnhollow Labs. Josiah Marchetti runs The Wrenford, the restaurant that felt the drift.
3 · THE OUTCOME
What does owning the shared prompt library well actually protect?
Tap to flip
ANSWER
That every team's guest-facing text stays reliable and calibrated as the underlying model changes, not just the one flow whoever built the library happens to be watching.
4 · THE RISK
What actually happens if an applied PM owns a shared, cross-team library?
Tap to flip
ANSWER
She optimizes it for her own feature's eval set. Every other team's prompts atrophy silently, and the longer it runs uncorrected, the more teams fork the drifted base, making it harder to untangle each month.
5 · THE GAP
What has to be true before any role variant can own this library well, and was it true at Tarnhollow Labs?
Tap to flip
ANSWER
More than one team actually using shared prompts, and a version-controlled eval set. The first was true for over a year. The second never existed, which is the real gap the ranking has to fix.
6 · THE NUMBER
Fill in the blank: "all set" language for elevated-risk reservations climbed from ___ percent to ___ percent across six weeks, and a broken shared prompt took ___ days to catch before platform ownership, versus ___ days after.
Tap to flip
ANSWER
3 percent to 15 percent. 42 days before, 2 days after platform ownership with a cross-flow eval set.
7 · THE RANK
State the final ranking of the four role variants, defended in one line each.
Tap to flip
ANSWER
Platform PM first, accountable to every flow. Applied PM second, a contributor who flags drift, never the owner. Infra PM third, keeps the pipe up, not the words calibrated. Research PM last, looks at the next model, not today's stable asset.
8 · CROSS-PRODUCT TRANSFER
Section 4 runs ORDER again on a different product. Which one, and what plays the role of the festival-Saturday-style miss there?
Tap to flip
ANSWER
Groundwork, Wickfield Civic Systems' inspection tool. A shared library's violation-notice flow started reading like a routine reminder after a phrasing update was tested on reminders only, so property owner Boniface Ekwueme missed a real deadline.

Check yourself Score: 0 / 0

Multiple choice
1. Why doesn't the fact that Wysteria's team built the prompt library first make them the right long-term owner?
  • A. Because Wysteria's team lacks the writing skill to maintain guest-facing copy.
  • B. Because an applied PM will naturally optimize the library for her own feature's eval set, leaving every other flow's prompts unwatched.
  • C. Because applied PMs are not allowed to touch shared infrastructure at Tarnhollow Labs.
  • D. Because the library was built before the waitlist and win-back flows existed.
Show hint
Check the Reversibility step in the ORDER recap.
Show answer
B. She's measured on her own feature's numbers, so that's what she checks. Nothing about who built it first changes that incentive.
Fill in the blank
2. The share of elevated-risk reservations texted "all set" language climbed from ___ percent in week 1 to ___ percent in week 6, while the blended no-show rate leadership watched only moved from ___ percent to ___ percent.
Show hint
Check the line chart, and the "what it costs at its worst" paragraph.
Show answer
3 percent to 15 percent. 10 percent to 11 percent. The blended number barely moved because elevated-risk reservations are only about a fifth of total bookings, so a real problem there gets diluted almost out of sight.
True or false
3. True or false: assigning the library to a platform PM alone, without building a shared eval set, would have been enough to fix this problem.
  • True
  • False
Show hint
Check the Dependency step, and the line about "ownership in name only."
Show answer
False. A platform PM title with no version-controlled test process underneath it is ownership in name only. Both the org-chart change and the eval set had to happen together.
Short answer, name the rejected alternative
4. Besides the other three role variants, what alternative does this answer name and reject, and why does it lose?
Show hint
Look at the "three things worth stating directly" paragraph near the end of the ORDER recap.
Show answer
Model answer: Letting each flow fork its own copy of the templates instead of centralizing ownership. It loses because forks drift further apart with every edit, which is a slower version of the exact failure that already happened once.
Short answer, apply it yourself
5. Think of an AI product you use or have built with more than one team's feature pulling from the same underlying prompt, model, or shared component. Who actually owns it today, and is that the role that should?
Show hint
Look for a piece of shared infrastructure that one team happened to build first, and ask whether anyone else is accountable to it.
Show answer
Model answer: A grocery app's item-substitution suggestions and its recipe-recommendation feature might both call the same shared "similar items" model. If the recommendations team owns it alone, substitution quality can quietly degrade as they tune it for recipes only, the same shape of drift as the prompt library here.
Short answer, work the number
6. If Placecard served 40 restaurants instead of 340, with the same one-third sharing the drifted template, would the same six-week drift still be worth fixing with a dedicated platform PM and eval set? Why or why not?
Show hint
Scale the 520,000 dollar platform-wide estimate down by the same ratio, and weigh it against what building the eval set costs.
Show answer
Model answer: probably yes, but the case gets less obvious. At 40 restaurants, the same math scales to roughly 61,000 dollars over six weeks, smaller, but still real money against a fix that costs a fraction of that in engineering time, and the risk compounds every quarter the library goes unowned, regardless of restaurant count.
Before you close the answer
Why this works
Tests whether you can rank real, structurally different roles instead of picking a favorite, and whether you can name what has to exist underneath a title before that title means anything. Most candidates can name a winner. Explaining why the other three genuinely lose, and what still needs building even after you pick right, is the part almost nobody does unprompted.
Follow-up traps
"Couldn't the applied PM just be more disciplined about testing every flow, without changing who owns it?" Response: discipline isn't a process. Nothing in her role makes her accountable to teams she doesn't report into or get measured against, so the incentive to skip it returns the next time she's under deadline pressure.

"Isn't handing this to platform just adding process for its own sake?" Response: no, the process is what caught the next migration's mismatch in 2 days instead of 42. That's not overhead, that's the entire fix showing up on time instead of six weeks late.
If pressed
The version-controlled eval set doesn't grade the new model against the old model's exact output. It grades each flow's output against an explicit, human-written contract per risk tier, "please confirm" must appear for elevated risk, "all set" must not, so a model that phrases things differently but still hits the contract still passes, and only a genuine miscalibration fails.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more