What safety requirements belong in every AI PRD regardless of feature?

Lumenreel streams licensed and original shows. Its AI recap tool writes the "previously on" summary that plays before a new episode. Amara Osei is the only person at the company whose job is to review every team's AI feature PRD before it ships, and she keeps a shared tablet on her desk that three different reviewers pass around during launch meetings.

How to answer this, stage by stage

The interviewer already knows "add safety requirements" is the right instinct. What they're testing is whether you can rank them under time pressure instead of listing five equally-weighted bullet points.

Let's learn

Say a company ships eleven different AI features across eleven different teams in a year. Each team writes its own PRD, and each one has its own idea of what "think about safety" means.

For a while, that worked out fine at Lumenreel. Amara would spot-check maybe every third PRD that crossed her desk, mostly the ones a team had already flagged as "sensitive" themselves. The ones that passed her spot-check looked genuinely careful. So she trusted the teams that didn't ask for a review to have made the same call correctly.

The recap tool was one of the eleven. It had shipped fourteen months earlier with no rollback plan and no named owner, because nobody on that team had marked it as a feature that "needed" a safety section, and Amara hadn't pulled it for a spot-check.

At its worst: the recap for a popular drama's season finale invented a plot detail that never aired, describing a side character as having cheated on their partner in an episode where nothing of the kind happened. Viewers screenshotted it as a "leaked spoiler" before the studio's own social team caught it and had to post a correction.

What I would leave alone: a feature like an AI-suggested watchlist ordering, where the worst case is a slightly worse row of recommendations, doesn't need the same five-item gate. Ranking the same five requirements as equally mandatory for every feature, regardless of what's actually at stake, would slow the harmless ones down for no real gain.

The lesson: a safety section that's optional by default isn't a safety requirement. It's a suggestion wearing a requirement's name.

Five things, and the recap tool's original PRD had a line about none of them.

Now here is the same thing as a story

The short version above is what you'd say defending this ranking on the spot. Read this one for how Amara actually found the gap.

Amara is the only person at Lumenreel whose full-time job is reviewing AI feature launches, across every team, not just her own. She's done it for three years, and for most of that time a spot-check felt like plenty.

She'd pull roughly every third PRD, mostly the ones a team had already flagged as touching something sensitive. The ones she checked were, almost without exception, thoughtful. So over time she stopped worrying much about the ones nobody flagged. If a team didn't think their feature needed a safety section, she trusted that judgment.

The recap tool's team never flagged it. Recaps felt low-stakes, a convenience feature, not a safety-adjacent one. It shipped fourteen months ago with a one-line "safety considerations: N/A" in a PRD nobody outside that team ever read closely.

This is the lifecycle Lumenreel has now. The recap tool, fourteen months ago, skipped straight from draft to launch.

Then came the near miss. A season finale recap, generated fresh for that week's new episode, described a fan-favorite side character as having cheated on their partner. It never happened in the show. The model had blended two separate subplots from earlier episodes into one invented detail, confidently, the way these models do when nothing is stopping them.

It went out to a small percentage of viewers before someone on the studio's social team, prepping the same week's promotional post, noticed the recap didn't match the actual episode and escalated it within the hour. Lumenreel pulled the recap and posted a correction before it spread much further. Genuinely lucky timing, nothing more.

Two of these four are cheap to fix after the fact. Two of them are not, and the recap tool had skipped both of the expensive ones.

When Amara pulled the recap team's PRD afterward, there was no named harm anywhere in it, so there had never been anything to build an eval set against. Building one from scratch, after the fact, meant going back through eight months of past recaps by hand to find the pattern, three full weeks of work that could have been two days if the harm had been named on day one.

We did not almost lose one recap. We almost let an invented plot detail travel further than a quiet correction could catch, and the only reason it didn't was a studio employee's unrelated Tuesday task.

The two hardest to retrofit are also the two that matter most. That's not a coincidence, that's why the rank exists.

The team rewrote the PRD template that month: every AI feature, no exceptions, states a named harm before a single design mock gets drawn. Ranked in the order they're hardest to add back: named harm, eval set, named owner, kill switch, disclosure.

Everything else in the pipeline depends on the first box. Skip it, and the rest can't really happen at all.

I used to think "safety considerations, if applicable" was a reasonable trust to place in eleven capable teams. It took watching one of them, in complete good faith, decide a recap tool didn't need it, to see that the word "applicable" was doing all the actual work, and nobody had ever defined it.

ORDER, the five rankedNot a checklist of five equal boxes. ORDER is what forces you to say which one you lose for good if you skip it.

The recap, one line per letter: outcome is viewer and studio-partner trust, reversibility is why named harm and eval set sit at the top, dependency is the eval set needing the named harm first, evidence is the cheap red-team pass that would have caught this, and rank closes with the actual five in order.

And if you want to be sure it really works, try it somewhere elseSame five letters, a gig-work marketplace instead of a streaming service. A completely different harm, and the hardest-to-retrofit item flips.

Sparefolk connects homeowners with freelance handypeople, and its AI tool drafts each job listing's description from a worker's photos and a short note about their experience. Ines Bouchard, who owns that feature, ran the same five requirements against it. The outcome: a homeowner's trust that the listing describes a worker's real qualifications. The named harm here isn't a hallucinated plot detail, it's the AI tool inventing a certification a worker never actually holds, since the model fills gaps in a thin note with plausible-sounding specifics. The hardest one to retrofit turned out to be disclosure, not the eval set: workers had already seen their AI-drafted listings go live for eight months with no label saying the description was AI-written, so freelancers had built their whole profile's voice around text they didn't write. Adding a disclosure label after that much time meant reopening a trust conversation with every worker on the platform at once, harder to undo than it looked on paper.

Five parts, and which one is hardest to fix later depends on the feature, not on the list itself.

Swap the trigger and it still runs.
Speed: an interviewer caps you at thirty seconds. Say "named harm and eval set first, because you can't rebuild those after launch," and stop.
Cost: there's no time to build all five before a hard launch date. Say so honestly, and ship with named harm and a named owner at minimum, since those two are the ones you genuinely can't add back later.
The model gets better, for real: if the recap model's overall accuracy improves, the named-harm eval set still matters just as much, since a rare, better-hidden hallucination is exactly what a coarse eval set misses first.

Where people run it wrong.
They write all five as an equally-weighted checklist, which tells a rushed team nothing about what to cut under pressure.
They rank by how scary a requirement sounds instead of by what's actually hardest to add back later.
They assume "if applicable" is a fine default, when in practice it just means whichever team is busiest decides for itself.

How to use it live. When someone asks what belongs in every AI PRD, ask yourself one question before answering: if we skip this one and launch anyway, can we still add it next month for cheap. Whatever answers no goes first.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0