Which traditional PM habits actively harm an AI product team? Name two and explain the damage.

Kettlecue is Grovewell's recipe recommender and shopping-list generator, built into a grocery delivery app. It reads what's in your cart, your stated diet and allergies, and what's on sale, then recommends a recipe and drops the ingredients straight into your list, plus small add-ons it thinks you'll want. Leopold Wrexham owns what counts as done for Kettlecue's features. Nnenna Adeyinka owns the model that pairs ingredients together. Cordelia Marrowby has a diagnosed sesame allergy, and she is the one who finds out what "done" actually meant.

How to answer this, stage by stage

Nobody is grading whether you can list good PM habits. They're grading whether you can name the exact sentence a habit turns into, and say who ends up stuck holding it.

Let's learn

Before Kettlecue, Cordelia Marrowby checked every ingredient list by hand, every single week, because a sesame allergy doesn't forgive a guess.

Kettlecue is Grovewell's recipe recommender: it reads what's in your cart and your stated diet, then recommends a recipe and drops the ingredients straight into your shopping list, including small "boost your bowl" add-ons it thinks you'll want on top.

Cross-checking a week of recipes against her own allergy list used to take Cordelia about twenty minutes every Sunday, reading every label, looking up unfamiliar ingredient names one at a time. Once Kettlecue's allergen filter proved itself, recipe after recipe, that dropped to about two minutes, just a skim. The filter earned that trust the slow way: Grovewell's content team had manually reviewed and tagged every official recipe's ingredient list against the top nine allergens for more than a year before Kettlecue ever launched.

Four things the add-on model checks. Allergens were never one of them.

Here's the turn. The recipe's own ingredient list was never the problem. It had been checked by a person for over a year. The problem was everything Kettlecue recommends on top of the recipe, the add-ons, which run through a completely different model that nobody ever pointed at the same allergy table.

Five clean demos in a row. Not one of them was the profile that mattered.

What it costs at its worst: Cordelia had a tahini drizzle suggested on top of a bowl recipe that was, on its own, completely sesame-free. She tapped it once to add it, cooked it that night, and had hives across her neck and arms within twenty minutes. An antihistamine handled it. No hospital visit. It was still a real allergic reaction, not a rounding error, and it happened because a recommendation, not a recipe, carried the allergen through.

When Grovewell pulled the last ninety days of "Boost your bowl" suggestions after the incident, about 3.4 percent of them, roughly 1 in 29, contained an ingredient that was a known synonym or derivative of a top-nine allergen with no allergen tag ever attached to it.

What I would leave alone: the recipe content team's manual review process is genuinely solid, a year of careful, human-checked tagging on every official recipe. I wouldn't touch it. The gap was never there.

The lesson: a spec that sounds strict isn't automatically safe. If nobody can tell you how the system would ever prove it met the rule, the rule is decoration, not protection.

Now here is the same thing as a story

Read the walkthrough above when the clock is running. Read this one when you want to feel what twenty minutes a week actually protects.

Cordelia Marrowby was diagnosed with a sesame allergy at eight years old, and by the time she was an adult she had a system: read every label twice, look up anything she didn't recognize, keep a running list of the words that meant trouble. Tahini. Benne. Halva. Gomashio. She never needed to guess, because she never let herself.

Kettlecue arrived the spring she signed up for Grovewell's delivery service, and for months it was exactly the relief it promised to be. She'd open the app, pick a recommended recipe, and the shopping list would just appear, already checked, already safe. The first few weeks she still read every ingredient on the recipe page out of habit. They were always clean. By the second month she'd stopped reading the recipe page at all, and by the third, she'd stopped noticing the difference between a recipe and one of the little "boost your bowl" suggestions Kettlecue tucked underneath it. To her, it was all just Kettlecue, and Kettlecue had never once been wrong.

Two people on the same feature. Only one of them ever got to decide what "done" meant.

Eight months before any of that, in a spec review nobody outside the team ever heard about, Nnenna Adeyinka had raised her hand. Leopold Wrexham had just finished reading out the acceptance criterion for the add-on model she owned: never suggest an ingredient on a user's allergy list, zero tolerance, no exceptions. She told him, as plainly as she could, that "never" wasn't something a model like this could promise. Ingredient names have synonyms, regional names, branded names. Tahini doesn't say sesame anywhere in its name. A rule with no room for a confidence score in it couldn't actually be tested as true, only as not yet caught being false.

Leopold's answer, more or less: just make it match the allergy list. That's the spec.

Nnenna built what she could build against a spec that admitted no partial credit: a keyword filter checking each suggested ingredient's name against the nine canonical allergen words. It passed every test Leopold ran against it, because every test used ingredients that spelled their allergen out directly. She raised the synonym gap once more, in writing, a month before launch. Nothing changed. Raising it a third time, this close to a ship date, would have read as her not being able to deliver, not as a live risk, so she let it go and moved to the next feature.

At the milestone review, Leopold demoed the add-on model live: five accounts he'd set up himself, vegetarian, keto, gluten-free, low-sodium, and one plain account with nothing flagged at all. Every add-on suggestion across all five looked appetizing and appropriate. He closed the risk section of the review with one line: "allergen handling, verified in demo." Nobody in the room asked whether any of the five accounts had an allergy that hid behind another word.

Four steps, one real gap. Nothing in the path ever routes an add-on through the check a recipe gets.

The Tuesday it mattered, Kettlecue recommended Cordelia a chickpea grain bowl, sesame-free, checked and clean, the way every recipe had been for months. Underneath it sat a small card: "Boost your bowl: silky tahini drizzle." She tapped it the way she tapped everything Kettlecue suggested by then, without reading it, because reading it had stopped being necessary a long time ago.

Twenty minutes after dinner, her neck and forearms were covered in hives. She took an antihistamine, watched herself carefully for an hour, and didn't need the hospital. She filed a support ticket that night at 9:40, still a little shaken, asking one plain question: how did an allergen get past a filter she'd trusted for months?

The investigation that followed found the gap fast, once someone finally looked: the add-on model Nnenna owned had never been connected to the same allergen tag table the recipe content team maintained by hand. It ran its own keyword filter, built eight months earlier against a spec that only understood the word "sesame," never the dozen words that mean the same thing.

In the retro, someone floated the fast fix: just add every synonym anyone could think of to the keyword list by hand. Tahini, benne, halva, gomashio. For about a day, that felt like progress. Then someone pointed out that Grovewell's catalog adds roughly 150 new items a week from partner brands, each with its own naming conventions, and a hand-built list would always be exactly one synonym behind, which is the precise failure that had just put Cordelia in the emergency-that-wasn't. They rejected it and built a graded ingredient matcher instead: a labeled set of real synonyms scored for confidence against the allergen taxonomy, checked by a food-safety reviewer, with any suggestion scoring under 0.9 confidence routed to a human before it could ship.

Run that Tuesday again with the matcher live. The tahini drizzle scores 0.97 against sesame the moment it's generated, gets held back automatically, and Cordelia never sees it as an option at all. Zero minutes spent wondering what happened, because nothing did.

One design hands the PM a pen and calls a demo proof. The other hands the engineer a real, checkable bar, and checks every surface against it, not just the one that used to get the careful attention.

What I'd tell myself, sitting in that spec review eight months earlier: the word "never" in an acceptance criterion isn't a safety feature. It's a promise nobody can keep, and somebody always ends up quietly building the closest thing they can, hoping the gap between the promise and the build never finds a real person.

GUARD, or who's left holding a feature nobody actually tested

Not a checklist for spotting risk in general. GUARD forces you to say, specifically, who absorbs the damage when a spec sounds strict but was never checkable.

One honest flag, raised once, quietly closed. Nothing about the process ever asked her to raise it again.

And if you want to be sure it really works, try it somewhere else

Same five letters, a school district instead of a grocery app, and the disguised allergen is a language gap instead of an ingredient name.

Sagebrook Unified School District runs Milemarker, a tool that flags students at risk of falling behind, using attendance and grade trends, and routes the flag to a counselor for outreach. Emmeline Larsson, the district's product lead, wrote its acceptance criterion the same way Leopold wrote his: "never flag a student who ends up graduating on time, full stop." No model can promise that as a hard rule, only as a calibrated rate on a labeled set. Her milestone demo ran on five students with long, stable local enrollment records, the easiest case Milemarker has. It never tested a new arrival with a thin record and a language gap, exactly where an attendance-and-grades model has the least signal to work with and the most room to be wrong.

Same shape of gap, a completely different siren. The record that goes thin here is a school history, not an ingredient name.

Same rank, different lever, mapped straight onto GUARD: the groups are the family behind the flagged student, and the counselor stuck acting on a number they can't verify either. The harm concentrates on new-arrival, thin-record families specifically, the group Milemarker's demo never included. The family has no clean way to contest a flag delivered in a language they don't share, and no data scientist on staff ever heard that concern raised, because nobody built the channel for it. The fix is the same shape: replace the zero-tolerance rule with a calibrated accuracy target checked against a labeled sample that actually includes thin-record students, and route every flag through a plain-language, translated appeal path before it reaches a family. Detecting it means tracking flag accuracy split by enrollment history length, not one blended district-wide number.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip to it: two ledgers, an eval spec instead of a fixed rule, and a representative sample instead of a demo, then watch for which surfaces never got wired to either one.
Cost: no budget this quarter for a full graded eval set. Start with the highest-risk surfaces, anything that lets someone tap straight to a cart or a decision, and grow the labeled set from there.
The model got better, for real: say Kettlecue's ingredient-matching model gets meaningfully more accurate overall. The eval spec still matters, because a better model just changes where the pass bar should sit. It doesn't remove the need for someone to keep checking it.

Where people run it wrong.
They read "zero tolerance" as more careful, when it's actually less checkable, since nobody can prove a probabilistic system will never do something.
They fix a bad demo by adding a sixth easy profile, instead of asking whether the sample is representative of who actually uses the product.
They watch one blended accuracy number and miss that the damage is sitting inside a single category underneath it.

How to use it live. Ask this before agreeing a spec is safe: "what's the graded, repeatable check for this, and did the demo include the hardest case, not just the easiest one?" That question alone usually tells you whether "it's tested" means a real eval spec or a demo that happened to go well.

Three things worth stating directly, since the real judgment sits here. Grovewell considered hand-expanding the keyword list with every synonym the team could think of, tahini, benne, halva, gomashio, instead of building a graded matcher. Rejected, because the catalog adds about 150 new items a week from partner brands, each with its own naming, so a hand-built list is always one synonym behind, the exact failure that let tahini through in the first place. The AI-specific failure worth naming is a grounding gap at the edge of a fixed taxonomy: matching free-text ingredient names and their regional or branded synonyms to a small, fixed allergen list is a language problem, not a lookup, and treating it like a solved lookup table is what let a real allergen through undetected. The guardrail is a confidence-scored ingredient matcher checked against a graded eval set, with anything under a 0.9 confidence score routed to a manual food-safety reviewer instead of auto-approved. And the trade-off is real and accepted on purpose: running that matcher on every real-time add-on suggestion, instead of a cached static tag, adds real cost and something like eighty to a hundred and twenty milliseconds to a "recommended for you" card that used to render instantly. Grovewell accepted that cost specifically on any surface that lets a customer add an item to their cart in one tap, and left pure browsing surfaces, like an inspiration article with no add-to-cart button, on the lighter, faster check.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0