ConceptIntermediateEval-Driven Specification / Writing a PRD for an AI feature / #20
What assumptions should be listed explicitly, and why does that matter more for AI?
The direct answer
List three kinds of assumption in every AI PRD: what the real input will actually look like, what your eval set counts as correct, and how the model should act when it isn't sure instead of always sounding certain. Regular software throws an error when its assumptions break. An AI feature just keeps answering, in the same confident voice, whether the input matches what it was tested on or not. Write these three down, or find out about the gap from someone who trusted the answer.
Do this, in order
Write down all three kinds of assumption in the PRD, not just the one that's obvious.Why: input shape, what counts as correct, and how the model handles being unsure are the three that break quietly in production, the kind a normal spec review never catches.
State exactly what the real input will look like, not what the test input looked like.Why: a tool tested against one shape of input will meet a different shape somewhere else, and nobody thinks to check unless it's written down as an assumption first.
State what your eval set treats as correct, and name what it never included.Why: an eval set built from one slice of the real world quietly excludes whatever case wasn't in that slice, and the score never says so.
Recut every launch metric by segment before calling it healthy.Why: a blended number can creep up by a fraction while one segment underneath it is running many times worse.
Run the one evidence test that separates an input problem from a real capability problem before fixing anything.Why: reshaping the same real requests into the tested format tells you whether the model is actually wrong, or just meeting input it was never built for.
How to answer this, stage by stage
Seven moves. The trap in this question is treating it as a list-the-risks prompt, so most of these stages exist to prove why an unwritten assumption behaves differently in an AI feature than a normal software bug does.
1
Ground it in one real feature and one number
Say it like this
"Let me put a number on this. Say a cafeteria company called Millrace Dining Group builds a tool called Trayline. A worker sets their diet once, and every week it reads that site's menu and tags each dish 'Safe for you' or 'Skip this one.' Bridget Okwuosa runs product for it. In the pilot, at one building on Fulton Street, Trayline's miss rate, saying a dish was safe when it actually had a restricted ingredient in it, sat at 0.3 percent."
Why this works
A real product, a real person, and a real number turn "name some assumptions" into something you can actually trace back to a decision.
2
Say your structure out loud
Say it like this
"I want to answer this like a diagnosis. Timeline, recut, assume nothing, cause candidates, evidence test. TRACE. The question sounds like it wants a list. What it's really asking is why a feature that tested fine can quietly stop being safe once it meets input it was never built for."
Why this works
Naming the plan up front tells the interviewer this isn't going to be five minutes of guessing out loud.
3
Say what the question is actually testing
Say it like this
"Here's the real test in this question. Normal software fails loud, a crash, an error you can see. An AI feature that hits an assumption it was never built for doesn't fail like that. It just answers anyway, in exactly the same confident tone, whether it's right or guessing."
Why this works
This sentence is the whole answer compressed. Skip it and everything after sounds like a checklist instead of a reason.
4
Give the three assumptions, straight
Say it like this
"So here's what I'd put in the PRD, in plain words. One, what the real input will actually look like, not what the test input looked like. Two, what our eval set counts as correct, and what kind of case it never included. Three, how the model should act when it's genuinely unsure, instead of always sounding just as sure."
Why this works
Naming three specific categories, not a vague "list your assumptions," is what separates a strong candidate from someone reciting good practice.
5
Walk the timeline, then recut it
Say it like this
"Trayline shipped to all sites in month one. A logistics site, Redmoor Logistics Park, came on in month four. A worker there got served a dish tagged safe that had a pork-derived stabiliser in the sauce, in month seven. The whole time, the company number barely moved, 0.3 up to 0.6 percent. So I recut it by site. Every office site was still under half a percent. Redmoor alone was running at 6.5."
Why this works
A number hiding thirteen times worse than it looks is the single strongest move in TRACE, and it's the one most answers skip.
6
Rule out the model, then run the one test that proves it
Say it like this
"Before I blame the model, I check whether it's actually the same version everywhere. It is. So I look at the input instead. Fulton Street's workers pick their diet from a dropdown. Redmoor's workers typed theirs into one open box, in a rush, on a shared kiosk. So here's the one test. Take Redmoor's real submissions, re-type them by hand into the same dropdown format Fulton uses, run the exact same model again. The miss rate drops from 6.5 to 0.8 percent."
Why this works
Same model, same dishes, only the shape of the input changed. That's what makes it evidence and not a guess.
7
Close on why this matters more for AI
Say it like this
"So that's the answer. A normal feature breaks in a way you can see. Trayline never crashed once. It just kept saying 'safe,' in the same confident voice, whether it was looking at input it understood or input it had never really been tested on. Write the assumption down, or find out about it from someone who trusted the tag."
Why this works
Ends on the decision, not a summary, and it's the line an interviewer actually remembers.
Let's learn
Trayline is a button inside Millrace Dining Group's cafeteria app. A worker sets their diet once, and every week it reads that site's menu and tags each dish "Safe for you" or "Skip this one."
Knowledge spark: what's a miss?
Trayline calling a dish safe when it actually has something in it a worker can't eat. Every miss it makes gets counted. The number everyone watches is how often that happens, out of every hundred tags.
Before Trayline, a worker with a restriction read the little ingredient card taped by each pan, every single day, or found a chef and waited. In the three-month pilot at the Fulton Street building, 640 workers signed up. Trayline's miss rate held at 0.3 percent the whole time. Nobody thought much about it. It just worked.
Millrace rolled it out to 14 sites over the next few months, about 9,200 workers in all. Company-wide, the miss rate crept up a little. Month one, 0.3 percent. Month three, 0.4. Month five, 0.5. Month seven, 0.6. Still a small number. Still looked fine on the weekly report.
The company-wide number, month by month
Flag-miss rate, blended across all 14 sites
m1, pilot pattern, 0.3%m3, more sites live, 0.4%m5, Redmoor 6 weeks in, 0.5%m7, worker served wrong dish, 0.6%
Here is the turn. That crawl from 0.3 to 0.6 percent is not the real problem, and it never was. The real problem showed up the moment somebody split that number apart by site instead of reading it as one company-wide average.
The number never lied. It just never once got looked at by itself, site by site.
At its worst, this costs Millrace a worker who trusted a tag with a real allergy or a real religious rule, served the wrong dish in a company cafeteria, in front of coworkers, by a tool built to protect exactly that.
The gap between when the real cause shipped and when anyone actually saw it
The choice I would take back. Bridget's team built Trayline's intake screen around dropdown chips, tested and tuned against exactly that shape of input, and never wrote down that this was an assumption rather than a fact. I would take that back. I would put the assumption in the PRD itself: this tool is tuned for structured input, not free text, and any site whose workers write their own words needs its own check before it ever goes live.
The decision that mattered
List the assumption instead of leaving it in one engineer's head. Input shape, what counts as correct, and how confident the model should sound, all three, written down, before rollout, not discovered from a complaint five sites later.
What I would leave alone. The 13 office sites using dropdown chips don't need any of this. Their input already looks like what the model was built and tested on, and their miss rate proves it. Rebuilding the intake for a problem that isn't happening there would just slow down a tool that's working.
The lesson. An assumption that only lives in one engineer's head at launch isn't a decision. It's a fact about the world that happened to be true on day one, and nobody wrote down that it might stop being true somewhere else.
The week Redmoor's tray line stopped looking like Fulton Street's
Read the short version above if you're short on time. This is the long version, for the part where you feel exactly what nearly went wrong.
Bridget Okwuosa spent two weeks in the Fulton Street cafeteria before she wrote a line of the PRD, standing by the register, watching people read the little cards taped to each pan.
She built Trayline the way she'd watched them already work. A worker sets their diet once, in a dropdown, picking every restriction that applies. Every week, the tool reads that week's menu and tags each dish. Safe for you. Skip this one.
For three months, it did exactly what it was built to do. Six hundred and forty people signed up. The miss rate sat at 0.3 percent and never moved. Bridget's team called the pilot done, and it was, for that one building.
So Millrace rolled Trayline out everywhere. Office towers first, then a distribution and logistics campus called Redmoor Logistics Park, in month four. Nobody changed the tool for Redmoor. Nobody thought they'd need to.
For a while, that looked right. The company-wide number crept, 0.4, then 0.5, then 0.6 percent, but it was still under one in a hundred. Nobody on Bridget's team looked at Redmoor by itself. Why would they. The whole picture still looked fine.
Then in month seven, someone on second shift at Redmoor got served a dish tagged safe with a sauce that had a pork-derived stabiliser hidden in the ingredient line, and dairy in it besides. She'd typed her restrictions into Trayline's one open box the week she signed up, in her own words, because the kiosk at Redmoor never had the same dropdown Fulton Street had. No pork, no gelatin, no alcohol, also lactose. All in one sentence.
We didn't build a worse tool for Redmoor. We built the same tool and quietly assumed the whole world would type like Fulton Street.
Bridget pulled the number apart that same week. Thirteen office sites, all under half a percent. Redmoor, alone, at 6.5. Same tool. Same model. One site had never once looked like the pilot.
She checked the model first, because that's the easy thing to blame. Same frozen version, everywhere, nothing had changed there. So she looked at what people had actually typed. Fulton Street's workers picked from a list. Redmoor's workers wrote sentences, in a rush, on one shared kiosk between two shifts, in a form nobody at Fulton Street had ever needed to use.
She took Redmoor's real requests and re-typed 380 of them, by hand, into the same dropdown chips Fulton uses. Same model. Same dishes. Only the shape of the words changed. The miss rate fell from 6.5 percent to 0.8.
So here is the decision I would take back. When we built Trayline's intake screen, we picked dropdown chips because they were fast to build and easy to test. That was the right call for a first site. What we never did was write down that this was an assumption, and not a fact, so nobody thought to check it again before the next site went live.
And the part I'd want to tell myself, if I could go back: we tested Trayline against the input we happened to have. We never once tested it against the input someone else might actually type.
What the recut actually showed
Before trusting the site-level gap, Bridget's team checked whether the tagging itself was even right. Two people hand-checked 20 of Redmoor's "Safe for you" tags against the real kitchen ingredient list. They agreed with the automatic miss flag on 19 of 20. The grading was not the problem. That left the input.
Same six months, cut by site instead of blended
0.3%
6.5%
13 office sites, blended dropdown-chip intake
Redmoor Logistics Park free-text intake, one kiosk
Sites using the tested input shape
The one site that never used it
The company-wide blend read 0.6 percent because Redmoor was only about 4 percent of everyone using Trayline. Its own number never showed up until someone cut it out on its own.
Redmoor, free text as submitted
380 real requests, one shared kiosk, month seven
6.5percent flag-miss, same frozen model
Redmoor, re-typed into dropdown chips
Same 380 requests, same dishes, same model
0.8percent flag-miss, once the input matched what was tested
Three assumptions, and the one that actually broke
Not because anyone was careless. Each of these, on its own, looks like a normal build decision. Together, they're the reason a feature that tested fine can quietly stop being safe.
Three separate, checkable assumptions, only one of them confirmed by the evidence test
Assumption 1
What the real input would look like.
Trayline was tuned against short, structured picks from a dropdown, one or two restrictions each, in English. Redmoor's workers wrote whole sentences into one open box, combining a religious rule, an allergy, and a preference in the same breath.
How you'd check it: re-encode a sample of the real free-text submissions into the tested format and rescore with the same frozen model. If the miss rate drops close to the pilot's number, this is the assumption that broke.
Assumption 2
What counts as correct.
The eval set that graded Trayline's tags was built entirely from Fulton Street's own approved menu, which never once contained a dish with a hidden animal-derived stabiliser. "Correct" was defined by a menu that had never contained the failure case at all.
How you'd check it: ask whether the eval set has even one example of the case that actually broke. If the honest answer is no, the eval set was never able to catch this, no matter how well the model scored on it.
Assumption 3
How often the model would be confidently wrong.
Trayline only ever prints two words, "Safe" or "Skip." During the pilot the wrong calls were rare enough that nobody asked for a third option. Once the sites got more varied, the same two words kept getting worn by dishes the model was actually guessing about.
How you'd check it: check whether the tool has any way to say "not sure, ask a chef." If every output looks equally certain, a wrong tag reads exactly like a right one.
TRACE, laid out across Redmoor's numbers
This reads like a question about listing good practice, but the real job is diagnosis: work out why a feature that tested clean can start failing in one corner of production without the aggregate number ever admitting it.
T, timeline. Trayline shipped to every site in month one. Redmoor Logistics Park came on in month four, with its own free-text intake box instead of Fulton's dropdown. Nobody flagged that difference at the time; it looked like a small kiosk detail, not a product decision.
R, recut. The same six months, split by site instead of blended. Thirteen office sites: all under half a percent. Redmoor alone: 6.5 percent. The company-wide average of 0.6 percent hid a site running roughly thirteen times worse than the rest.
A, assume nothing. Before blaming the input, rule out the model and the grading. Same frozen version deployed everywhere, no separate build for Redmoor. Two people hand-checked 20 of Redmoor's "safe" tags against the real kitchen ingredient list and agreed with the automatic flag on 19. That rules out a broken model and a broken measurement, and leaves the input as the only variable still standing.
C, cause candidates. Three, named and separate: the input shape, dropdown chips assumed while free text arrived; what "good" means, since the eval set never had a dish like the one that reached Redmoor; and confidence, since Trayline only ever says safe or skip, never not sure.
E, evidence test. Re-type Redmoor's 380 real submissions into the same dropdown chips Fulton uses, and score them with the exact same frozen model. The miss rate falls from 6.5 to 0.8 percent. Same model, same dishes, only the input's shape changed, which is what proves it's the input-shape assumption and not a model that can't reason about halal and kosher combinations.
Why the evidence test is the hard step
Anyone can suspect the input caused it. The evidence test turns that suspicion into two numbers off the same model, one for the real free text and one for the reshaped version, and shows exactly how much of the gap the input's shape explains, instead of a hunch dressed up as a finding.
Same blind spot, a permit photo nobody checked in daylight
The City of Larchmere's building department runs a tool that reviews photos of home electrical panels against code and auto-approves the ones that clear it. Odera Nwachukwu leads product for it. It launched for licensed contractors first, then opened to homeowners doing their own small permits five months later.
T. The auto-approval tool launched for licensed contractors in month one. Homeowner permits opened in month five. The blended miss rate, wrongly approving a panel that should have been flagged, crept from 0.4 to 0.6 percent over the next three months, and nobody split it apart until Odera pulled the numbers in month eight. R. Recut by who submitted the photo. Contractor photos: 0.4 percent, steady the whole time. Homeowner photos: 9 percent. A. Same frozen model scored both groups. A manual review of 15 flagged homeowner photos agreed with the tool on 14. The grading held up. C. Three candidates, the same shape as before: photo lighting and angle, since contractors shoot with a proper phone rig in good light and homeowners shoot in a dim basement holding a flashlight; what "good" means, since the eval set that graded the tool was built entirely from contractor photos; and confidence, since the tool only ever says approved or flagged, never hold for review. E. Splitting the 310 homeowner photos by how well lit they were was the check. The well-lit ones, about 40 of them, scored close to the contractor rate. The dim ones carried almost the entire gap, which pointed straight at the photo itself, not at homeowner wiring being genuinely worse.
Swap the trigger and it still runs
Speed: Millrace could have rolled out to all 14 sites in one week instead of four months, to hit a launch date. TRACE still starts by asking what shipped and when the miss actually reached someone, not by how fast the rollout happened.
Cost: leadership could have cut the segment-level QA pass to save two weeks before launch. The recut step still has to happen eventually, just after a complaint instead of before one.
The model really did get better: say Trayline's next version genuinely got sharper everywhere, fewer misses at every office site, in the very same stretch Redmoor's number got worse. TRACE still finds Redmoor, because the recut isolates one site even while the overall trend looks like good news.
Where people run it wrong
Trusting a blended number that's still technically small, without ever cutting it apart by segment.
Treating one wrong tag as proof the model needs more training data, before checking whether the input even matched what it was tested on.
Fixing the visible symptom, retraining on more religious-diet examples, instead of the actual gap, the intake box itself.
How to use it live
Buy yourself ten seconds by naming the split out loud. "So there's the number everyone's watching, and there's whatever that number is quietly averaging together. Let me say how I'd check whether one segment is carrying the whole thing." That's not stalling. That's where the real diagnosis starts.
Flashcards (click a card to flip it)
This is a concept question about naming assumptions, worked as a diagnosis, so these eight test the TRACE moves and the real numbers behind them.
1 · THE FRAMEWORK
Which framework fits "what assumptions should be listed explicitly," and why?
Tap to flip
ANSWER
TRACE. It sounds like a request for a list, but the real job is diagnosis: working out why a feature that tested fine can quietly stop being safe once it meets input nobody planned for.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Bridget Okwuosa, product manager for Trayline, Millrace Dining Group's dietary-restriction meal-plan tool. She built and ran the successful three-month Fulton Street pilot herself.
3 · THE HABIT
What did the team stop doing once the pilot succeeded?
Tap to flip
ANSWER
Checking whether the assumptions behind the pilot still held at the next site. The blended, company-wide number looked fine, so nobody looked at any one site by itself for six months.
4 · THE THREE ASSUMPTIONS
Name the three kinds of assumption that matter more for AI than for regular software.
Tap to flip
ANSWER
What the real input will look like, what the eval set counts as correct, and how the model should act when it's genuinely unsure instead of always sounding certain.
5 · THE NUMBER
Redmoor Logistics Park's miss rate sat at 6.5 percent while the company-wide blended number only read ______ percent.
Tap to flip
ANSWER
0.6 percent. The blend hid it because Redmoor was only about 4 percent of everyone using Trayline, so its bad number barely nudged the average.
6 · THE CHECK
Name the one test that proved it was the input, not the model.
Tap to flip
ANSWER
Re-typing Redmoor's real free-text requests into the same dropdown chips Fulton Street uses, then running the exact same frozen model. The miss rate fell from 6.5 to 0.8 percent.
7 · THE FIX
What should the PRD have said, before Trayline ever reached a second site?
Tap to flip
ANSWER
That Trayline was tuned for structured, dropdown-style input, and any site whose workers type in their own words needs its own check before launch, written down as an assumption instead of left in one engineer's head.
8 · CROSS-PRODUCT TRANSFER
Section 4 runs TRACE again on a different product. Which one, and what's the number?
Tap to flip
ANSWER
The City of Larchmere's panel-photo approval tool. Contractor photos scored 0.4 percent miss, homeowner photos scored 9 percent, and splitting homeowner photos by lighting showed the well-lit ones close to the contractor rate.
Check yourself Score: 0 / 0
Fill in the blank
1. Trayline's miss rate at Redmoor Logistics Park was ______ percent, while the company-wide blended number only read ______ percent.
Show hint
Look at the recut chart, the two bars split by site, next to the blended line chart above it.
Show answer
6.5 and 0.6. The gap only showed up once someone cut the number by site instead of reading the company-wide average.
Multiple choice
2. Bridget's team built Trayline's intake screen around dropdown chips instead of free text. Was that decision reckless?
A. Yes, dropdown chips are always the wrong choice for restriction input.
B. No, dropdown chips were the sensible, fast choice for the pilot. The problem was never writing down that this was an assumption, not a fact that would hold everywhere.
C. Yes, because it caused the miss rate to climb to 6.5 percent at Redmoor.
D. No, because dropdown chips have no effect on the model's accuracy.
Show hint
Ask whether the choice itself was wrong, or whether it was the lack of a written-down assumption that caused the real problem later.
Show answer
B. The dropdown was the right call for a first site. What broke things was never writing down that it was an assumption about input, not a guaranteed fact about every future site.
True or false
3. True or false: since the company-wide miss rate only crept from 0.3 to 0.6 percent, that proves the model itself was getting worse at reading restrictions.
True
False
Show hint
Look at which single thing was held constant across the whole six months.
Show answer
False. The same frozen model ran the whole time. The recut and the evidence test both point to the input's shape at Redmoor, not the model getting worse.
Short answer
4. Name a place at Millrace where this same input-shape fix would not matter, and say why.
Show hint
Think about a site whose workers already submit their restrictions the way the model was built and tested on.
Show answer
Model answer: "Leave the 13 office sites alone. Their workers already use the same dropdown chips Trayline was built and tested on, and their miss rate proves it, all under half a percent. Rebuilding intake there would spend effort on a gap that doesn't exist."
Short answer, apply it yourself
5. Think of an AI feature you use yourself. What's one assumption about the input it was probably tested on that your real use might not match?
Show hint
Look for a case where what you feed it doesn't quite match the clean, structured version it was probably tuned on.
Show answer
Model answer: "A resume-screening tool was probably tuned on resumes typed in one language and one format. A resume scanned from a printed page, or written with different formatting habits, might quietly get read worse, even though the tool never once says it's unsure." Any honest answer works if it names a real gap between the tested input and the input you actually give it.
Multiple choice
6. If re-typing Redmoor's requests into dropdown chips had left the miss rate at 6.5 percent instead of dropping it, what would that have shown?
A. That the input shape was probably not the problem, and the deeper issue was more likely the model's own reasoning about halal and kosher combinations, or a real gap in what the eval set considered correct.
B. That the evidence test itself was broken and needs to be re-run.
C. That Fulton Street's dropdown chips were also unsafe.
D. That the company-wide blended number was accurate all along.
Show hint
If reshaping the input hadn't closed the gap, ask what's left standing as an explanation.
Show answer
A. The evidence test only confirms the input-shape hypothesis because reshaping the input closed most of the gap. If it hadn't, the input would be ruled out, pointing back at the model or the eval set instead.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
