U2xAI
…
ConceptIntermediateResponsible AI & Advanced Practice / Responsible AI as a product requirement / #1

How do you turn a responsible AI principle into a testable product requirement?

LEAD the product is Verve, a short-video app with an AI system that ranks comments under every post

Verve is a short-video app. Its comment-ranking model decides which replies show up first under a video, including which ones climb into the Top 10 a creator actually reads. Dax Okonkwo runs trust and safety for that ranking system, and keeps a laminated one-page checklist taped to the edge of his monitor.

The direct answer
Write the principle as a number tied to a check, not a value statement. Verve's rule "we don't amplify harassment" becomes: no more than 5 percent of comments that three or more people report may still sit in a video's Top 10 an hour after posting, measured against a held-out harassment eval set on every ranking model change, with the launch blocked automatically if it fails.
Do this, in order
  1. Turn the principle into one number tied to a check, not a value statement.Why: a principle nobody can test never actually gets tested, no matter how often it's repeated in a deck.
  2. Build the held-out eval set before you write the threshold.Why: a threshold with no real set behind it is a guess wearing a number's clothes.
  3. Pick a leading signal, not the harm itself.Why: if you wait for real harm to show up in a support ticket, the harm already happened.
  4. Make the check block the launch automatically.Why: a check a person can wave through under deadline pressure isn't a check, it's a suggestion.
  5. Name one way the number gets gamed, and watch for that specifically.Why: every metric has a cheap way to look good without anyone doing the real work.
  6. Re-test the eval set itself as the product's comment shapes change.Why: a set built for last year's harassment patterns goes stale quietly, and a stale set is worse than no set because it looks like proof.

How to answer this, stage by stage

The interviewer isn't grading whether you can recite "responsible AI" as a phrase. They're grading whether you can turn it into something an engineer could actually fail a build against.

Stage 1
Ground it in one real feature
Say it like this
"I'll answer this for Verve's comment-ranking system, since that's where a vague safety principle actually has to turn into code."
Why this works
Stops the answer from floating in the abstract space where every "responsible AI" question tends to drift.
Stage 2
Name your structure
Say it like this
"I'll use LEAD. Link it to the real outcome. Find the early signal. Name how it gets gamed. Say what decision each level of that signal actually triggers."
Why this works
Tells the interviewer you're not about to give a values speech, you're about to build a spec.
Stage 3
Say what the principle actually means
Say it like this
"'We don't amplify harassment' really means: our ranking model should never be the reason a pile-on comment gets more eyes than it would have gotten on its own."
Why this works
Translates a value into a mechanism before a single number gets attached to it.
Stage 4
Give the one number
Say it like this
"No more than 5 percent of comments reported by three or more people may still sit in the Top 10 an hour after posting. Checked against a held-out set on every model change."
Why this works
This is the actual answer to the question. Everything before it was setup.
Stage 5
Prove it with the near miss
Say it like this
"Here's why that number matters: a coordinated pile-on almost made it into a creator's Top 3 comments, and the only reason it got caught was a moderator happened to be looking, not because anything was built to catch it."
Why this works
Compresses the story into four sentences instead of retelling the whole thing.
Stage 6
Say what would still slip through
Say it like this
"This threshold catches slurs and obvious pile-ons. It won't catch coded harassment with no bad words in it, so I'd pair it with a human-reviewed sample, not replace review with the number entirely."
Why this works
Shows you know a single number is never the whole defense, which is exactly what the "abuse" step in LEAD is for.
Stage 7
Close on the number and the block
Say it like this
"So: 5 percent, checked every launch, and the launch doesn't ship if it fails. That's the whole requirement."
Why this works
Ends on the same sentence you opened with, which is what makes an answer feel finished instead of trailing off.

Let's learn

Say a company writes a value into its charter: we don't amplify harassment. Everyone nods. Nobody can tell you what would make that sentence false.

Verve's comment-ranking model used to get spot-checked by hand. Once a quarter, someone on the trust and safety team would pull ten ranked comment threads and read through them, looking for anything that felt like a pile-on had climbed too high. It always looked fine. So the checks got shorter, then less frequent, then folded into a slide nobody presented anymore.

Knowledge spark: what's a held-out eval set? A pile of real examples nobody trains the model on, kept aside just for testing. If you test using the same data the model learned from, you're really just checking its memory. A held-out set checks whether it actually learned the right thing.

There was no single bad quarter that started this. The number that mattered, the share of reported comments still sitting in a video's Top 10 an hour later, drifted upward for six straight weeks while nobody was watching it, because nobody had ever named it as a number worth watching.

The leading signal, six quiet weeks
12% 6% 0 Week 1 Week 3 Week 5 Week 6: 11%, near miss
Nothing in Verve's launch dashboard moved during these six weeks. The dashboard tracked engagement, not this.

At its worst: a coordinated pile-on comment climbed to third place under a creator's video, ahead of hundreds of ordinary replies, and stayed there for forty minutes before a moderator who happened to be scrolling that day caught it.

The decision I would take back We wrote "we don't amplify harassment" as a value in the launch doc and left it there, because at the time nobody could think of a clean number that captured it and the team had a launch to hit. That was fine when the ranking model rarely touched anything close to the edge. It stopped being fine once ranking got good enough at surfacing "engaging" replies, since a pile-on is, mechanically, extremely engaging.

What I would leave alone: a single blandly negative comment sitting low in the rankings doesn't need this kind of gate. Not every unkind reply is a pile-on, and treating every low-rated comment as a harassment signal would bury real, if harsh, feedback along with the actual danger.

The extra reports were never really the problem. The problem was that "we don't amplify harassment" had no number attached to it, so nothing could ever tell us we'd broken our own promise.

The lesson: a principle with no number behind it isn't a value the company holds, it's a sentence the company hopes stays true.

Hand sketched icon list titled What makes a principle testable. Five items: a gauge icon labeled a named number not a feeling, a document icon labeled a real eval set behind it, a box icon labeled a version pin on the model, a scale icon labeled a named owner who gets paged, a funnel icon labeled a block not just a warning.
Five things, and Verve's original launch doc had exactly zero of them.
Hand sketched metaphor scene titled A value versus a check. Left panel, a document icon labeled Principle, caption we do not amplify harassment. Right panel, a gauge icon labeled Requirement, caption cap it, check it every launch.
A principle is a document nobody can fail a test against. A requirement is a number with a gate behind it.

Now here is the same thing as a story

The short version above is what you'd say cold in an interview. Read this one for how the near miss actually happened.

Dax has run trust and safety for Verve's comment system for four years. He's the only one at the company whose whole job is watching what the ranking model does to a thread, not just whether creators are posting.

When the ranking model first shipped, harassment complaints were rare enough that a quarterly hand-check felt like plenty. Dax would pull ten threads, read them over coffee, and email a one-line summary to the team: looked fine, nothing to flag.

It built up slowly, over about six weeks, and nobody noticed because nobody had a number pointed at the right thing. The ranking model had gotten better at predicting which replies would get more responses, which meant it had also gotten better, completely by accident, at surfacing the exact kind of comment a coordinated pile-on produces: short, reactive, and dozens of people replying to each other in a chain.

Hand sketched timeline titled Six quiet weeks. Four milestones. Week 1, 2 percent, normal. Week 3, 5 percent, nobody looked. Week 5, 8 percent, still no alarm. Week 6, 11 percent, near miss caught, emphasized.
Four weeks of drift with nothing watching it, because the thing worth watching had never been named.

Then came a Tuesday. A creator posted a video about a local news story, and within twenty minutes a coordinated group of accounts had replied to each other with the same accusatory comment, worded just differently enough each time to avoid Verve's word-filter. The ranking model, reading all that reply-to-reply activity as high engagement, pushed the comment to third place under the video.

Dax happened to be scrolling that creator's page for an unrelated reason. He saw the comment sitting at third place, checked the report queue, and found three people had already flagged it. Nobody had told him. Nothing had told him. He just happened to be looking.

He pulled the ranking logs for the past six weeks that same afternoon, and built the number for the first time: the share of reported comments still sitting in Top 10 an hour after posting had gone from 2 percent to 11 percent, quietly, while every dashboard the team actually watched stayed flat.

Hand sketched quadrant titled Verve's principles, before and after. Axes how measurable from a feeling to a number, and how bad if broken from minor to severe. No amplify harassment before sits top left, severe and a feeling. No amplify harassment now sits top right, severe and a number. Fast load times sits bottom right. Friendly tone sits bottom left.
The principle didn't get less important. It moved from a feeling everyone shared to a number somebody could fail.

We did not almost lose a launch that day. We almost let a coordinated pile-on outrank hundreds of real replies under a stranger's video, and the only reason it didn't ship to more people was luck, not design.

The team wrote "no more than 5 percent, checked every launch" that same week, and built the eval set behind it from six months of past reported comments, tagged for whether they'd stayed in Top 10.

Hand sketched flow diagram titled From a value to a gate. Five boxes: value written, eval set built highlighted, threshold set, checked at launch, blocks bad launch.
The eval set is the step everyone skips, because it's the slow one. It's also the one the whole gate depends on.

I want to say the model got worse. It didn't, not really. It got better at exactly the thing it was told to optimize, engagement, and a pile-on is engagement wearing a bad costume. That's the part a value statement can never catch, because a value statement doesn't know what the model is actually being scored on underneath it.

We wrote "we don't amplify harassment" into a launch doc two years ago because it was true, and true felt like enough. It took a stranger's pile-on almost reaching third place under a video to see that a sentence nobody can fail isn't a promise. It's a hope with good intentions.

LEAD, in one screenNot "how sure is the model." How early would this number have told us, and what do we do at each level of it.

L
Link. The real outcome.
Creators stop posting when their comment sections turn hostile. That's the business cost underneath "we don't amplify harassment."
Names the outcome the principle is actually protecting, not the model's own score.
E
Early signal. The thing that moves first.
Percent of reported comments still in Top 10 after one hour. It drifted from 2 to 11 percent over six weeks while every dashboard the team watched stayed flat.
The hardest step, and the actual answer to the question: this is what "testable" means in practice.
A
Abuse. How it gets gamed.
A model tuned only to catch slurs would pass this check easily while missing coded, no-bad-words pile-ons entirely.
Every metric has a cheap way to look good. Naming it here means someone has to check for it later.
D
Decision. What happens at each level.
Under 5 percent, ship. Between 5 and 8, ship with a flagged sample for human review. Above 8, the launch blocks itself, no human override.
A metric nobody acts on is a chart on a wall. This is what makes it a requirement.
How long a reported comment stayed visible, before and after the gate
0 47 min Before the gate 6 min After the gate
The number the early signal was quietly predicting: how long a reported comment gets to sit in front of everyone before anything happens.

The recap, one line per letter: link is creator retention behind the harassment promise, early signal is the reported-comment-visibility rate that moved six weeks ahead of anything else, abuse is a coded pile-on that dodges a slur filter, and decision is the three-tier response from ship to auto-block.

And if you want to be sure it really works, try it somewhere elseSame four letters, a handmade-goods marketplace instead of a comments feed. A completely different harm, and this time the input mix is the trick.

Almscroft Market lets independent makers list handmade goods, and an AI tool drafts each listing's description from a few uploaded photos and a short seller note. The company's principle is "we don't let AI listings mislead a buyer." Priya, who owns that feature, mapped it onto LEAD like this. Link: buyer trust in the marketplace, since a buyer burned once by a misleading AI description stops buying from anyone on the platform, not just the one seller. Early signal: the return rate specifically on AI-drafted listings for items over 100 dollars, since that's where sellers started rationing which items they'd trust the AI tool with. Abuse: a seller could pass this check by only ever using the AI tool on cheap, low-risk items, which makes the eval set look great while telling you nothing about the expensive listings buyers actually complain about. Decision: below a 4 percent return-for-mismatch rate, ship as-is; above it, require a seller to confirm three AI-drafted claims by hand before the listing goes live.

Hand sketched labeled parts diagram titled What is inside the requirement. Center document icon labeled Testable Requirement, with four callouts: eval set, threshold, owner, block trigger.
Four parts, and the eval set is the one that took the longest to build and mattered the most.

Swap the trigger and it still runs.
Speed: an interviewer gives you thirty seconds. Say "turn the value into a number, build the eval set first, and make the number block the launch," and stop there.
Cost: there's no headcount to build a full held-out set this quarter. Say so honestly, and start with the twenty worst historical cases as a rough set rather than none at all, since a rough gate beats a hope.
The model gets better, for real: if the ranking model's overall relevance score improves, that's exactly when this check matters most, because a smarter model is also a smarter pile-on amplifier, not a safer one by default.

Where people run it wrong.
They write the value statement and call it done, since it feels complete on the page.
They pick the harm itself as the metric, which means by definition the harm already happened before the number moved.
They build a threshold with no eval set behind it, so the number is really just a guess with a percent sign on it.

How to use it live. When someone hands you a values statement and asks you to make it real, ask yourself one question first: what would move, quietly, weeks before anyone could actually see the harm. Name that thing out loud before you touch a single threshold.

Flashcards (tap any card to flip it)

1 · THE FRAMEWORK
What framework fits "how do you turn a responsible AI principle into a testable requirement"?
Tap to flip
ANSWER
LEAD: link it to the real outcome, find the early signal, name how it gets gamed, decide what happens at each level.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Dax Okonkwo, who runs trust and safety for Verve's comment-ranking system and keeps a laminated checklist taped to his monitor.
3 · THE OLD HABIT
What did the team stop doing because it kept looking fine?
Tap to flip
ANSWER
A quarterly hand-check of ten comment threads, which shrank to nothing because it never found anything, until nobody was watching the right number at all.
4 · THE EARLY SIGNAL
What's the leading indicator in this story, and why that one?
Tap to flip
ANSWER
Percent of reported comments still visible in Top 10 after one hour. It moved for six weeks before anything else did.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Leaving "we don't amplify harassment" as a value statement in the launch doc instead of a number, since no clean number existed yet and there was a launch to hit.
6 · THE NUMBER
Fill in the blank: the leading signal drifted from 2 percent to ___ percent over six weeks.
Tap to flip
ANSWER
11 percent. The new requirement caps it at 5 percent, checked on every launch.
7 · THE REPLAY
Same coordinated pile-on, redesigned gate. What changes?
Tap to flip
ANSWER
The launch that would have amplified it never ships. A reported comment's median time visible in Top 10 drops from 47 minutes to 6.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what's the gaming risk there?
Tap to flip
ANSWER
Almscroft Market's AI listing generator. There, sellers could game the check by only using the AI tool on cheap items, hiding the real risk on expensive ones.

Check yourself Score: 0 / 0

Multiple choice
1. Why does this answer pick "percent of reported comments still in Top 10 after one hour" instead of just counting harassment complaints directly?
  • A. Complaints are too hard to collect from users.
  • B. Because it moves weeks before real harm shows up, while a complaint count only moves after the harm already happened.
  • C. Because complaints are always false.
  • D. Because reports are legally required to be tracked separately.
Show hint
Look at the line chart of the six quiet weeks.
Show answer
B. A leading signal is the whole point of LEAD's E step: it should look healthy right up until the morning things actually break.
True or false
2. True or false: this answer recommends replacing human review entirely with the 5 percent threshold.
  • True
  • False
Show hint
Look at "what would still slip through."
Show answer
False. The threshold catches obvious pile-ons, but coded harassment with no bad words needs a human-reviewed sample alongside it, not instead of it.
Fill in the blank
3. Fill in the blank: the median time a reported comment stayed visible in Top 10 dropped from 47 minutes to ___ minutes after the gate shipped.
Show hint
Look at the bar chart comparing before and after the gate.
Show answer
6 minutes. That's the lagging outcome the early signal was quietly predicting six weeks in advance.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at "the decision I would take back."
Show answer
Model answer: Leaving the harassment principle as a value statement with no number. It made sense while ranking rarely touched anything close to the edge, and no clean number existed yet.
Short answer, where it wouldn't matter
5. Name a kind of comment where this gate genuinely does not need to apply.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: A single blandly negative comment low in the rankings. Not every unkind reply is a coordinated pile-on, and treating every harsh comment as a harassment signal would bury real feedback along with real danger.
Short answer, apply it yourself
6. Pick an app you use that has some kind of written value or promise attached to it (privacy, fairness, safety). What's one number you could build that would move before that promise actually gets broken?
Show hint
Think about what a person would start doing differently right before the promise breaks, not after.
Show answer
Model answer: For a privacy promise, a good early number is how often a feature accesses data it doesn't strictly need for that session, tracked before any leak happens, the same shape as this answer's reported-comment rate.
Before you close the answer
Why this works
Tests whether you can turn an inspiring but untestable value into a real, gameable-but-monitored number, instead of stopping at the value statement itself, which is where most candidates stop.
Follow-up traps
"What if 5 percent is just the wrong number, too strict or too loose?" Response: it's a starting point built from six months of past reported comments, not a guess, and it gets re-tested against fresh data as comment patterns shift, the way any threshold should.

"Doesn't auto-blocking the launch just slow the team down constantly?" Response: only when the number is actually bad, and a launch that would push a pile-on into Top 3 is exactly the launch that should be slowed down.
If pressed
The held-out eval set behind this threshold pins the exact ranking model version it was tested against, since a model that gets quietly updated invalidates the whole test without anyone noticing.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more

More on Responsible AI as a product requirement