The direct answer
Write the content policy as a named table of disallowed categories, each one with a real line that passes and a real line that fails, not a sentence like "keep it appropriate." Build the table around whoever actually ends up hearing or reading the output, not whoever typed the prompt. Then keep sampling what actually shipped against that same table after launch, because the gap always shows up in production, not in the design review.
Do this, in order
Write the policy as a named table of disallowed categories, each with a real pass example and a real fail example.Why: this is the one document decision the whole answer turns on.
Build the table around whoever ends up exposed to the output, not whoever requested it.Why: what counts as harmful depends on who's on the receiving end, not who typed the prompt.
Give the categories that land worst in one specific context the most concrete examples.Why: those are exactly the ones a vague line like "be appropriate" never catches.
Keep sampling what actually shipped against that same table after launch, not just once at model-tuning time.Why: a policy nobody rechecks quietly stops matching what the model is actually doing.
Give the person exposed to a bad output a way to flag the exact line, not just "report a problem."Why: without a specific flag, the same gap keeps shipping to everyone else too.
Leave harmless plot inconveniences off the list entirely.Why: naming everything as risky buries the categories that actually matter.
How to answer this, stage by stage
Seven moves. Naming both audiences, showing where a gap lands hardest, and writing the actual table line are where the real answer lives.
1
Ground it in one real product before naming a framework
Say it like this
"Say we build an app that writes a new bedtime story every night. A parent picks a theme, a lost puppy, a brave bunny, taps generate, and the app either reads it out loud or the parent reads it, right at bedtime. The question is what 'appropriate for children' actually means, written down in a way an engineer, or a model, can be checked against."
Why this works
Grounds a policy question in one real product before naming a method, so the answer can't stay abstract.
2
State your structure in one line
Say it like this
"I'd use GUARD here, because a content-policy question is a risk question wearing a spec question's clothes. Who's exposed and who just typed the prompt, where a gap in the rule lands hardest, who can't tell it slipped through, the actual rule you'd write down, and how you'd catch it once it's live."
Why this works
Two seconds that show a plan exists before the interviewer hears you improvise a list of good intentions.
3
Name both audiences, not just the one who typed the prompt
Say it like this
"There's the parent, who picks the theme and taps generate, usually with half an eye on their phone. And there's the child, five or six years old, lying in the dark, who hears every word of whatever gets generated and has no idea it came from a model at all. The parent authorized the story. The child is who it actually happens to."
Why this works
This is GUARD's G step, naming both sides before proposing anything.
4
Show where a missing rule lands hardest, with one example
Say it like this
"Most generated stories are fine at any hour. A bunny loses a carrot, that lands the same whether it's noon or midnight. But a story where a puppy gets separated from its family and the ending never says they found each other, that's a completely different thing read out loud in a dark room to a five-year-old than it is sitting in a design review at two in the afternoon."
Why this works
This is U. It shows harm doesn't land evenly, with one concrete example instead of a general warning.
5
Name who can't flag it before it spreads
Say it like this
"The child can't tell anyone the story scared her, not in a way that reaches an engineer. And even when a parent does flag it, if the report just says 'bad story,' with no line and no category, it sits in a queue while the same gap keeps generating for every other family typing in 'lost puppy' that same week."
Why this works
This is A, GUARD's hardest step, and the one most policy docs skip.
6
Write the actual rule, not a values line
Say it like this
"Here's what I'd put in the document. A named list: no separation from a parent or guardian without a reunion inside the same story, no injury beyond a mild scrape, described exactly, no fear imagery held past one sentence. Each line gets one example that passes and one that fails, right next to it, not in a separate appendix nobody reads."
Why this works
This is R, and it's the actual answer. It names specific categories, not a blanket rule for everything.
7
Say how you'd catch a gap once it's live, then close
Say it like this
"I'd pull a real sample of what actually shipped, every week, and score it against that same list by hand, because the gap always shows up after launch, not during it. That's the whole answer. Write the categories down with real examples, build them around who actually hears the story, and keep checking what shipped against the same list, not just what you meant to ship."
Why this works
This is D, plus a close that restates the decision in one breath.
Let's learn
What happens when the feature that's supposed to save a parent fifteen minutes a night quietly decides, on its own, what a five-year-old sits with in the dark?
Say we build an app that writes a new bedtime story every night. A parent picks a theme, a lost puppy, a brave bunny, an under-the-sea adventure, taps generate, and either reads the story to their child or lets the app read it out loud.
Knowledge spark: what's a safety rule for a generative feature?
A written line that says exactly what a model is never allowed to produce, specific enough that a person, or the model itself, can check a real piece of output against it and get a clear yes or no.
Before the story generator, a parent spent about 15 minutes some nights digging through a shelf of books, looking for one their child hadn't heard forty times already. With the generator, that drops to under a minute. Pick a theme, tap generate, done.
In the app's first two months live, the team previewed every story the model generated before it ever reached a child, and nothing close to scary made it through. Parents got used to it working. Most stopped reading a story before playing it. They trusted the app the same way they trusted a book off a shelf.
Here is the turn. Once the model got good enough to invent its own plot instead of leaning on a handful of pre-checked ones, roughly one story in every eleven, about 9 out of every 100 sampled later, separated a character from its family and never brought them back together by the end. No swear word. No monster. The story just ended sad, in a way built for a bedtime read to a five-year-old alone in a dark room. Nobody had written a rule against that, because "keep it appropriate for children" never said what a five-year-old, specifically, at 8pm, specifically, can't sit with.
It wasn't a bad word that scared her. It was a story that never said the puppy came home.
At its worst, this costs the app the exact trust it was built to earn. A parent who gets woken at 2am by a crying child stops letting the app pick the story at all. They go back to the shelf, the same 15 minutes a night, except now they also have to explain why the story made their kid cry.
The decision I would take back
We wrote the content policy as one line: keep it appropriate for children. That was fine when the model mostly repeated a handful of pre-checked plots. It stopped being fine the moment the model got good enough to invent its own stakes, because "appropriate" never said what a stake even means for a five-year-old at bedtime.
What I would leave alone. A character getting a tummy ache from too many berries, or losing a mitten in the snow, needs no special rule. Small plot trouble is most of what makes a story worth hearing. Only the categories that land differently in the dark, at night, alone, need to be named.
The lesson. A rule nobody can check a real story against isn't a rule, it's a hope. If you can't point to the exact line that broke the policy, the policy was never really written down.
Now here is the same thing as a story
Read the short version above if you're short on time. Read this one when you want to feel why the rule matters, not just know what it is.
Elin Bosch has run product for the story generator since before it had a single user, back when the whole app was three pre-written stories and a name field to fill in.
For its first eight weeks live, she read every story the model generated before it was ever queued to a family, forwarding herself the day's batch each morning over coffee, sometimes eighty stories in one sitting. A dragon scared of the dark. A cat who lost her hat. Sweet, small, exactly what the app promised.
Same story, same phone. Only one of them ever gets read.
By the third month, she'd stopped reading the batch daily. The stories kept being fine, and she had a hundred other things to build. She dropped to reading a sample twice a week, then once a week, on Fridays, alongside the metrics review. Then she mostly just watched the star rating parents left after each story, steady at 4.8 for weeks, and let that stand in for reading it herself.
Nobody told her this was a problem. Every week, the numbers said it was working.
Then came an email from support, not flagged urgent, just odd: a parent writing at 6am, saying her daughter had woken up crying at 2am because "the puppy story never said the puppy came home."
Elin pulled the story. Theme: lost puppy, one of the most requested in the app, generated fresh every time. This version had the puppy get separated from its family in a storm, wander for three pages, and the story just ended, mid-search, on a page about the puppy being tired and alone. No line about anyone finding anyone. The model had written a genuinely well-crafted, unresolved story, and shipped it at 8:41pm to a five-year-old.
That weekend Elin pulled a real sample to see how big the gap actually was: 500 stories generated across four themes that same fortnight. Forty-six of them, about 9 out of every 100, ended the same way a scary moment never got resolved. Broken out by theme, the lost puppy stories were the worst of the four by a wide margin.
Stories with no resolution to a scary moment, by theme
500 stories sampled the same week as the incident, before the named categories shipped.
Under the sea adventure
7%
The 4.8-star average never showed this. A story can be well told and still leave a child alone with an animal that never got found.
It wasn't a bad word that scared her. It was a story that never said the puppy came home.
I want to say the problem is the model got worse at writing endings. It didn't, really; if anything it got better at building real stakes into a three-minute story. But Elin never had a number in her head either. She had a feeling: this app writes fine stories, or this app needs watching. A rating steady at 4.8 never once told her the feeling should have flipped.
Back when the team wrote the very first content guidelines, in a single afternoon before the beta opened, the whole policy was one line in the doc: "keep all generated content appropriate for children." Someone in that meeting asked if they should list out anything specific. The answer, at the time, was no, the model only had three plots to work with, there was nothing specific yet to list.
I would go back and write the list anyway. Even with three plots. Especially with three plots, because that's the cheapest moment to write down what "appropriate" excludes, before the model is doing something interesting enough to need it.
Run the same fortnight again with one line added to the document: no separation from a parent or guardian without a reunion inside the same story. The model gets that as an actual instruction, not a vibe. Elin reruns the prompt that made the 2am story. This time the puppy finds its way home by the second-to-last page, licks someone's face, the end. Total time to check that against the rule: eleven seconds, because now there's something specific to check it against, not a feeling.
What I'd tell the version of myself in that first meeting: "we'll know it when we see it" isn't a policy. It's a promise that somebody will always be looking. Eventually nobody is.
Spelling out GUARD for a story nobody previewed
This is a risk question, so the framework is GUARD. "Document the guardrails for a generative feature" reads like a spec-writing question, which is exactly why one vague line looked like a real policy until a support email found the gap.
G, groups. The parent, who picks the theme and taps generate. And the child, who hears every word of whatever gets generated and has no idea it came from a model at all.
U, unequal. A silly plot inconvenience, a lost mitten, a tummy ache, lands the same at any hour. A story that leaves a character separated with no resolution lands completely differently read aloud in the dark at bedtime than it does in a review queue at two in the afternoon.
The step that should sit before the same gap ships to the next family.
A, ability to contest. A five-year-old can't file a report on which line of a story scared her. And a parent's flag that just says "bad story," with no line and no category, doesn't stop the same gap generating for every other family typing "lost puppy" that same week.
R, reduce. A named list, in the document: no separation from a parent or guardian without a reunion in the same story, no injury beyond a mild scrape, described exactly, no fear imagery held past one sentence. Each line paired with one example that passes and one that fails.
D, detect. Pull a real sample of what actually shipped, every week, and score it against that same list by hand. A star rating that stays steady never tells you a story left someone alone in the dark. Only reading it does. Four weeks after the fix, the same sample came back under 1 in 100 across every theme.
Where this answer would fail
If the fix is a longer values statement, or telling the model to "use good judgment" about scary content, none of it counts. A named category with a real pass and fail example, and a weekly sample scored against that same list, are build items with an owner and a cost. Somebody can put them on a roadmap this quarter, and you can check whether they shipped.
And if you want to be sure it really works, try it somewhere else
A funeral home's website offers a tool that drafts a first pass of an obituary from notes a family submits, meant to be reviewed and edited before it's approved. Different industry entirely, same five letters, same gap.
G, groups. The funeral home staff, who enter the family's notes to get a draft started quickly. And the family, exhausted and grieving, who read the finished version, along with every guest who hears it read aloud at the service.
U, unequal. A slightly generic sentence about "a life well lived" is harmless. An invented detail, a wrong date, a guessed cause of death, a made-up anecdote presented as fact, is a completely different thing once it's printed in a program and read aloud to a room of people who knew the person.
A, ability to contest. A grieving family often approves a draft without reading every line closely. They can't flag a wrong phrase they never fully absorbed until it's already printed and being read at the service.
R, reduce. A named list: no invented anecdote presented as fact, no assumed cause of death, no assumed religious or cultural language unless the family provided it. Each line paired with a real example of a draft that passes and one that doesn't.
D, detect. Sample a share of drafts before they're marked "sent to family," checking for invented specifics, and track how many corrections come in after printing, since that's the signal something slipped through unnoticed.
Swap the trigger and it still runs
- Speed: a faster model can write a story in under a second, and that speed is exactly what makes it tempting to skip the weekly sample, because nothing about "fast" tells you it's still safe.
- Cost: a cheaper model comes in at a third of the price, and the team runs ten times more volume for the same review budget, right up until nobody notices the sample rate quietly dropped back to what it always was.
- The model gets better: the story model gets good enough to build real emotional stakes into a bedtime story on purpose, because it makes for a better story, and that exact improvement is what breaks a one-line policy that only ever assumed simple plots.
Where people run it wrong
- Writing "use good judgment about scary content" once, in a values doc, with no actual list a person or a model could check a real output against.
- Treating a steady star rating as proof nothing's wrong, when a rating only tells you the story was liked, not what a five-year-old sat with alone at bedtime.
- Naming the categories once at launch and never rechecking them after the model changes, gets retrained, or a new theme gets added.
How to use it live
Ask, out loud: "if this went out tonight, who actually hears it, and could they tell anyone if it went wrong?" If you can't answer both halves for the feature in front of you, the policy isn't finished. Say that, then name the one category from this story that's missing from whatever example you've been given.
Flashcards (click a card to flip it)
1 · THE FRAMEWORK
Which framework fits a question about documenting guardrails and content policy for a generative feature, and why?
Tap to flip
ANSWER
GUARD, for risk. The real question isn't what words to put in a values statement, it's who's exposed to the output, where a missing rule lands hardest, who can't flag it before it spreads, and how you'd catch a gap once it's live.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Elin Bosch, the product manager who built the bedtime story generator. She used to read every story it made before any child heard one.
3 · THE HABIT
What did Elin stop doing once the app had been live a few months?
Tap to flip
ANSWER
Reading a sample of generated stories herself. She dropped from daily, to twice a week, to just watching the star rating, which stayed steady the whole time.
4 · THE GAP
What's the two-number gap this whole answer turns on?
Tap to flip
ANSWER
9 out of every 100 sampled stories ended with no resolution to a scary plot point (a lost puppy never found), against under 1 in 100 once the named categories and weekly sampling were in place.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Writing the content policy as one line, "keep it appropriate for children," instead of a named, checkable list. It made sense when the model had three fixed plots. It stopped making sense once the model could invent its own stakes.
6 · THE NUMBER
Fill in: a parent's daughter woke up crying at ______ because the puppy story never said the puppy came home.
Tap to flip
ANSWER
2am. The parent didn't find out why until she read the story herself the next morning and emailed support.
7 · THE REPLAY
Same "lost puppy" theme, new policy in place. What changes?
Tap to flip
ANSWER
The model gets the actual rule: no separation from a parent or guardian without a reunion in the same story. The puppy finds its way home by the second-to-last page. Checking it against the rule takes eleven seconds, because now there's something specific to check.
8 · TRANSFER
Section four runs GUARD again on a different product. Which one, and what does the reduce step become?
Tap to flip
ANSWER
A funeral home's obituary drafting tool. Reduce: no invented anecdote presented as fact, no assumed cause of death, no assumed religious language, unless the family actually provided it.
Check yourself Score: 0 / 0
Short answer
1. Why doesn't "keep it appropriate for children" work as a content policy, even though every person on the team already knows what it means?
Show hint
Think about what a person can check against versus what a model, or a reviewer sampling its output later, can check against.
Show answer
Model answer: "Because a person can read 'appropriate' and know instantly what it excludes, but a model producing a new story every time has to be told the actual boundary in a way it, or a reviewer, can check afterward. 'Appropriate' has no example of a failing line in it, so nobody, human or model, can point to the moment it was broken."
Fill in the blank
2. In the sample review, ______ out of every 100 stories, across all themes, ended with no resolution to a scary moment, before the named categories shipped.
Show hint
It's the overall figure behind "the gap" flashcard, not the lost-puppy theme's own higher number.
Show answer
9. 46 out of 500 sampled stories. The lost puppy theme alone ran much higher, 18 in 100, which is exactly why that theme was the one that produced the incident.
Multiple choice
3. Which old decision does this answer take back?
- A. Writing the content policy as a single line, "appropriate for children," instead of a named, checkable list.
- B. Letting the app generate a new story every night instead of reusing a fixed set.
- C. Adding a weekly sample review after the incident.
- D. Telling the model to write shorter stories.
Show hint
Look for the decision made back when the team first wrote the guidelines, not the fix proposed after the incident.
Show answer
A. C is the fix, not the reversal. B gives up on the feature instead of designing something safer. D is unrelated, a dial nobody proposed turning. Only A names the actual choice, one vague line standing in for a real policy, that this answer undoes.
True or false
4. True or false: once the app started sampling shipped stories weekly against the named categories, the risk was gone for good.
Show hint
Ask what happens the next time the model changes, gets retrained, or a new theme gets added.
Show answer
False. Sampling only catches what's already shipped since the last check. If the model gets retrained, or a new theme launches, the same kind of gap can open again, which is why detection has to keep running, not run once.
Multiple choice
5. If the unresolved-separation rate had been much lower, say 2 in 100 instead of 9 in 100, what's the strongest reason it would still belong in the named list?
- A. It's cheap to write down, so there's no real cost to naming every category, common or not.
- B. A lower rate still means an actual child, on an actual night, hears an unresolved story. The category's value is about how it lands, not how often it happens.
- C. A rare category is usually a sign the sample itself was flawed, not a real gap.
- D. It wouldn't. Only categories above roughly 5 in 100 are worth writing into the policy.
Show hint
Ask whether this category's value comes from how often it happens, or from how badly it lands the one time it does.
Show answer
B. Frequency changes how urgently you'd fix a gap, not whether it belongs on the list at all. A and D treat naming as a cost-benefit call on volume; C dismisses a real finding as noise. Only B ties the category's worth to what happens to the one child it lands on.
Short answer, apply it yourself
6. Pick a generative feature you've used yourself, one that produces new content each time. What's one category of output it should probably name and check against, but likely doesn't?
Show hint
Look for the kind of output that's fine in most contexts but lands badly in one specific one.
Show answer
Model answer: "A resume-summary tool that also drafts a one-line 'why you'd be a great fit' blurb. It's fine when the blurb stays vague and safe, but it should specifically name and check against 'never invents a specific past employer or number that wasn't in the source resume,' because a hiring manager reading it assumes every detail came from the applicant, not the model."
