The direct answer
Write three fallback states into the spec, not one generic "something went wrong" screen: the model can't be reached, the model answered but its own confidence is under the line you set, and the model flagged an emergency pattern. Each state needs its own wording and its own next action, and the low-confidence state should point at a human, not a retry button. Never let a low-confidence answer render exactly like a confident one, and don't have the app quietly retry the model in the background on day one.
Do this, in order
Write three distinct fallback states, each with its own message and next action.Why: this is the whole requirement, one banner can't do the job of three different kinds of failure.
Route the low-confidence state straight to a human contact, not a retry button.Why: a shaky answer sitting on screen for a minute delays a call that has real hours ticking against it.
Never let a low-confidence answer render exactly like a confident one.Why: the dangerous failure is the one that looks fine, not the one that visibly breaks.
Skip silent auto-retry on day one.Why: a background retry hides the minutes passing, and a second try can quietly hand back a different answer with no explanation.
Leave the plain features' generic error banner exactly as it is.Why: a booking screen failing costs five seconds, it doesn't need three states.
Have someone actually read a sample of low-confidence responses every week.Why: the gap here surfaced from a review, not a dashboard, and dashboards go quiet exactly when nobody's looking at them.
How to answer this, stage by stage
Seven moves. Ground it in one app's own booking screen before you touch the AI feature at all.
1
Scope it to one AI feature that already sits next to a plain one
Say it like this
"Say I'm the PM on a pet health app. There's a booking screen that's been fine for two years, and a symptom checker built on a model that shipped fourteen months ago. I'll write the error-handling requirements against that actual pair, not against AI features in general."
Why this works
One real pair, one boring and one risky, gives the interviewer something to check your answer against, instead of a policy that could apply to anything.
2
Say your structure out loud
Say it like this
"Here's how I'll walk through it: what a normal error already gets right, why that same fix breaks for a model, the actual states I'd write into the spec, what I'd leave out on purpose on day one, and how I'd know it's working."
Why this works
Two seconds of structure tells the interviewer you have a plan before a single detail lands, so they aren't guessing where you're headed.
3
Reframe what an AI error actually is
Say it like this
"Most error handling means the request failed. For a model, the dangerous case is the opposite. It doesn't fail. It answers, and it's wrong, and a calm wrong answer looks exactly like a calm right one on the same screen."
Why this works
This is the actual insight the question is testing. Skip it and you've written requirements for outages, not for the model.
4
Give the anchor: three states, with real copy
Say it like this
"So here's the three states I'd write in. Can't reach the model: say so, offer one manual retry. Answered under our confidence line: say so plainly, put the vet's number one tap away. Flagged an emergency pattern: skip the soft language, name the next action right now, no retry button at all."
Why this works
Naming three states with actual words on them is something an interviewer can picture as a real spec section, not a promise to "handle errors gracefully."
5
Prove it with a failure, in four sentences
Say it like this
"Here's what happens without that middle state. A dog's showing symptoms that could be nothing or could be an emergency. The model isn't sure, answers anyway, and the screen shows it exactly like a confident answer. The owner waits until morning instead of calling that night."
Why this works
A specific failure with a real cost persuades in a way "this could go wrong" never does.
6
Say what you would not build on day one
Say it like this
"I wouldn't have the app silently retry the model in the background and just show whatever comes back. That hides the minutes passing, and a second try can hand back a different answer with no explanation for why it changed. If it can't answer with confidence, say so, don't quietly try again and hope."
Why this works
Shows judgment instead of a wish list, and it answers a follow-up almost every interviewer asks next: "why not just retry?"
Say it like this
"So: three states, each with its own message and its own next action, no silent retries, and the low-confidence state routes straight to a human, not back into the model. If a requirement can't say what the screen shows the exact moment the model isn't sure, it isn't finished yet."
Why this works
Interviewers remember the last line most, and this one hands them a concrete test they can run on any AI feature's spec afterward.
If you remember one thing
Stages 4 and 6 are the answer. The three named states that make the screen honest, and the explicit refusal to fix this with a silent retry loop. Everything else here is proof it works.
Let's learn
Say we build an app for pet owners called Whisker Watch. Type in what's wrong with your dog, and a model sorts it into one of three calls: watch it, call the vet this week, or call the vet now.
The app already has a plain feature that fails sometimes: booking a vet appointment. About 1 in 40 booking attempts hiccups on the first try. The owner sees one line, "Something went wrong, please try again," taps it, and it works. Total cost: five seconds and no worry, because a booking failure is never actually urgent.
What a normal error looks like today, before any model is involved
Knowledge spark: what triage means here
Sorting symptoms into three piles: watch it at home, call the vet soon, or call an emergency vet now. A nurse hotline does this over the phone. The symptom checker is trying to do the same sorting with a model instead of a person on the line.
The symptom checker breaks differently, and it doesn't always look like breaking at all. In a thirty day sample of five hundred conversations, forty one came back with the model's own confidence sitting under the cutoff the team uses internally to mean "don't say this alone." All forty one showed up on screen exactly like the other four hundred and fifty nine: one calm paragraph, no flag, nothing different at all.
Low-confidence answers that got a different message than a confident one
Before this spec (0 of 41)
After this spec (41 of 41)
Forty one conversations a month came back with the model unsure. Before this requirement, every one of them looked exactly like a confident answer. After it, every one gets its own message and a vet's number, every time.
Here's the part that's easy to miss.
None of those forty one technically errored. The screen had nothing to apologize for. The model answered, it just wasn't sure, and the app never said so.
The model didn't crash. It answered, calmly, and got it wrong, and nothing on the screen said otherwise.
At its worst, this costs an owner a full night. They read "keep an eye on him for now" at eleven at night, close the app, and don't call anyone until morning, for a set of symptoms that needed a phone call that same night.
The decision that mattered
Write three fallback states into the spec, not one. Can't reach the model: say so, offer a manual retry. Answered, but not confident: say so plainly, put a vet contact one tap away. Flagged an emergency pattern: skip the soft language, name the next action now.
The choice I would take back. Months earlier, the PRD's error-handling section was one line: "Standard error handling applies," pointing straight at the booking screen's banner. That banner had worked for two years, and writing three separate messages felt like effort nobody had asked for.
What I would leave alone. The booking screen doesn't need any of this. If it fails, a five second retry is the whole story, because nothing about picking a new appointment time is urgent. Leave it exactly as it is.
The lesson. An AI feature's error handling isn't the rest of the app's error handling wearing the same three sentences. It's the part of the spec that decides what a person believes the moment the app isn't sure, and it has to be written on purpose, not inherited from a screen that never had to carry that weight.
Now here is the same thing as a story
Read this one when you've got three minutes, since eleven o'clock on a Wednesday says more than any spec section ever could.
Odalys Ferreira has run product for Whisker Watch's care team for a bit over a year. Before the symptom checker, she owned the booking flow: the calendar, the reminders, the apology banner that shows up when the server hiccups. She wrote that banner herself. It works. Nobody thinks about it.
The symptom checker shipped fourteen months ago, and for most of that time it did exactly what it promised. Type in what's wrong with your dog, get back one of three calls. Odalys read a sample of the transcripts every Friday afternoon, forty or so, checking the model's tone and its calls against what a vet would actually say.
She kept that up for about two months.
Then she started only reading the ones the model itself had flagged as unclear. Then she started skimming those. Then, most Fridays, she didn't open the file at all, because nothing bad had turned up in months and three other launches were sitting on her desk.
The anchor: three named states, not one generic screen
On a Wednesday morning, a one-star review came in from Mette Sorensen, whose eleven-year-old Labrador Otis had spent the night pacing, drooling, and trying and failing to be sick. She'd typed all three symptoms into Whisker Watch at half past nine. The app told her to keep an eye on him and check again in the morning. She did. By seven, Otis still hadn't settled, and the emergency vet down the street said the words that scared her more than anything the app had: "that's what bloat can look like, and bloat can kill a dog in hours." It turned out to be gas. The vet was clear it easily could not have been.
Odalys pulled the actual conversation. Nothing had crashed. No banner, no apology, no retry button, because from the app's point of view nothing had gone wrong. The model had answered. Buried in a log only engineers ever opened, its own confidence on that answer sat well under the line the team used to mean "don't say this alone." On the screen, none of that showed. It read exactly like the four hundred and fifty nine confident answers sitting next to it that same month.
The day the model wasn't sure, with and without the anchor in place
She pulled the last thirty days to see how often this happened. Five hundred conversations. Forty one came back under that same confidence line, and all forty one had shown up looking identical to a confident answer, calm paragraph, no flag, nothing to tell Mette or anyone else that the app wasn't sure.
We didn't need the model to stop being wrong. We needed the app to stop sounding sure when it wasn't.
Odalys never thought in confidence numbers before that Wednesday. She thought in exactly two settings: the app has something to say, or the app is having trouble and admits it. There was no third setting for "I said something, but you shouldn't lean on it alone," and that missing setting was the whole gap.
She remembered the actual meeting where this got decided. Eight months earlier, scoping the symptom checker's spec, the error-handling section was one line: "Standard error handling applies," pointing at the same banner from the booking flow. Nobody argued with it. That banner had worked for two years. Writing three separate messages for a feature that, on paper, was just another form on a screen, felt like effort nobody had asked for.
Now run the same Wednesday night forward with three states instead of one. Mette types the same three words at half past nine. The model's confidence lands under the line, same as before. This time the screen doesn't hand her a calm paragraph. It says, plainly: "We're not confident enough to tell you this is nothing. Here's what to watch for tonight, and here's the after-hours line for your registered vet," with the number one tap away. She calls at nine forty. The vet on the phone asks two questions and tells her to come in to be safe. It's still gas. But the call happens at twenty to ten, not seven the next morning, and the ten hours in between stop being a gap nobody meant to leave open.
One design hands her a paragraph and hopes she reads the tone right. The other hands her a phone number.
And the thing I'd tell myself, if I could go back to that scoping meeting: we wrote pages about what the model should say when it's sure. We never wrote a single line about what happens the day it isn't, and that's exactly the day the feature exists for.
SPARK, five settings for the day the app isn't sure
This question asks for a spec section, not a screen, but it's still a design decision about what a person sees the exact moment the product has nothing solid to say, so SPARK still fits. A question asking how to measure the symptom checker's quality would reach for LEAD instead.
SPARK, aimed at a requirements section instead of a screen
S, situation. A Whisker Watch owner today, on any feature that isn't the symptom checker: a booking screen fails, they see one line and a retry button, tap it, and move on in five seconds.
P, payoff. Not "fewer complaints." The habit worth building: an owner who sees the app admit it isn't sure trusts it more the next time it does answer confidently, because silence has never once hidden a bad moment from them.
A, anchor. Three fallback states written into the spec, not one: can't reach the model, answered but under the confidence line, flagged an emergency pattern. Each with its own message and its own next action.
R, risk. The AI fails in a way no generic error message covers. It doesn't crash, it answers wrong, calmly, sitting right next to four hundred and fifty nine right answers that looked exactly the same.
K, keep out. No silent auto-retry on day one. A background retry hides the minutes ticking by, and a second attempt can hand back a different answer with nothing on screen explaining why the story changed.
Why the anchor has to survive the risk
Check it against the Wednesday night story: does the three-state screen still work the exact moment the model is unsure and wrong? Only if the low-confidence state fires before anyone reads a wrong answer as a right one. An anchor that only handles "the server is down" hasn't touched the actual risk, it's solved the easy failure and left the dangerous one exactly where it was.
What day one leaves out, on purpose
And if you want to be sure it really works, try it somewhere else
A city's recycling app is a different department entirely, and the same three-state gap shows up there too, just wearing a different bin.
Same framework, a different city department, a different bin
S. Elham Sadeghi leads product for Kerbwise, the city's app where a resident photographs an item and a model says which bin it goes in. Today, without the model, residents call the city's waste line and read the sorting guide out loud to whoever answers.
P. The habit worth building: a resident who gets an unclear photo result still knows exactly what to do next, instead of guessing and putting something in the truck that contaminates the whole load.
A. Three states again, same shape, different content. Can't process the photo, too blurry or dark: ask for a clearer shot. Processed it, confidence under the line: say so, show both likely bins. Flagged a hazardous item, batteries, propane, paint: skip the retry, send straight to the drop-off point.
R. A blurry photo of a lithium battery gets a calm, wrong "recycling" label, sits in the truck with everything else, and starts exactly the fire the ops team was already worried about before anyone wrote a line of the spec.
K. No silent auto-retry on the photo either. If the model can't tell, ask the resident to take a clearer shot themselves. Don't quietly re-run the same blurry photo three times and guess again.
It took a text from the city's contamination inspector, not an outage, to get this written down: "you know it keeps telling people batteries are fine to bin, right?"
Swap the trigger and it still runs
- Speed: even if the model answered in a tenth of a second, a fast wrong answer is still wrong. Three states don't disappear because the model got quicker.
- Cost: if running the model got free tomorrow, that still doesn't tell a resident what to do differently with a hazardous item. Cheap confidence isn't the same as correct confidence.
- The model gets better: if accuracy climbs to 99 percent, the one wrong answer left in a hundred still needs its own state. A rarer failure isn't a smaller one for the person holding it.
Where people run it wrong
- Writing one error section that only covers "the model is unreachable," and calling the spec finished.
- Treating a low-confidence answer as a smaller problem than an outage, when it's the one nobody notices until it's too late.
- Fixing this with a silent retry loop instead of a message, so the user never even learns the app struggled.
How to use it live
If you're asked this cold, ask what a normal error screen already does in this product before inventing anything new. Then ask what happens the moment the model is technically right there and wrong anyway. That second question is where the real requirements live.
Flashcards (click a card to flip it)
1 · THE SITUATION
What does the plain, non-AI version of this feature do when it fails?
Tap to flip
ANSWER
Whisker Watch's booking screen shows one line, "Something went wrong, try again," and a retry button. About 1 in 40 attempts needs it, it works in seconds, and nobody thinks about it again.
2 · THE REFRAME
Why can't an AI feature reuse that same one banner?
Tap to flip
ANSWER
Because the dangerous AI failure isn't a crash. The model answers, calmly, and gets it wrong, and that looks exactly like a right answer on the same screen.
3 · THE ANCHOR
What three fallback states does the spec require?
Tap to flip
ANSWER
Can't reach the model, say so and offer a manual retry. Answered under the confidence line, say so and put a vet contact one tap away. Flagged an emergency pattern, skip the soft language, name the next action now.
4 · THE RISK
What breaks the first time the model fails in a way no generic message covers?
Tap to flip
ANSWER
It answers wrong, calmly, and nothing on screen tells the owner to double check anything, so a call that needed to happen that night waits until morning instead.
5 · THE PROOF
What actually happened the night Mette typed Otis's symptoms into the app?
Tap to flip
ANSWER
The model's confidence sat under the internal cutoff, but the screen showed a calm "keep an eye on him" with nothing different from a confident answer. She waited until morning to call, for symptoms that looked like a possible emergency.
6 · THE NUMBER
___ of the 500 symptom conversations came back under the confidence line, and before the fix, ___ of them looked any different on screen.
Tap to flip
ANSWER
41 of 500 (about 8 percent), and 0 of them looked any different. Every low-confidence answer had shown up looking identical to a confident one, until the requirement changed that.
7 · THE REPLAY
Same Wednesday night, new requirement. What changes?
Tap to flip
ANSWER
The low-confidence state fires instead of a calm paragraph. Mette gets a message saying the app isn't confident, plus her vet's after-hours number one tap away. She calls at twenty to ten instead of seven the next morning.
8 · CROSS-PRODUCT
Section 4 runs SPARK again on a different product. Which one, and what does its anchor add?
Tap to flip
ANSWER
Kerbwise, a city's recycling photo app. Its anchor keeps the same three states, and routes a flagged hazardous item, batteries or propane, straight to "send to the drop-off point," skipping the retry entirely.
Check yourself Score: 0 / 0
Fill in the blank
1. In the thirty-day sample, ___ of the five hundred symptom conversations came back under the app's confidence line, and before the fix, ___ of them looked any different from a confident answer on screen.
Show hint
Look at the numbers Odalys found once she finally pulled the sample after Mette's review.
Show answer
41, and 0. All forty-one low-confidence answers had shown up looking identical to a confident one, which is exactly the gap the three-state requirement closes.
True or false
2. True or false: the safest fix for a low-confidence AI answer is to have the app quietly retry the model in the background until it returns a more confident answer.
Show hint
Think about what a silent retry hides from the person waiting on the answer.
Show answer
False. A background retry hides the minutes passing and can quietly swap in a different answer with no explanation, which is worse for trust than plainly saying the app isn't sure.
Multiple choice
3. Which of these belongs in the AI feature's error-handling requirements, according to this answer?
- A. One message covering every failure, so the spec stays short.
- B. A distinct message for a low-confidence answer that routes straight to a vet contact.
- C. A silent retry loop that keeps trying the model until it returns a clean answer.
- D. Nothing extra. The booking screen's banner already covers this.
Show hint
Ask which option a user could actually act on the moment the app isn't sure.
Show answer
B. A and D collapse three different failures into one message, which is the mistake this answer argues against. C is the keep-out this answer explicitly rules out for day one.
Short answer
4. What old decision does this answer take back, and why did it make sense when it was first made?
Show hint
Think about what made reusing the booking screen's banner feel like the obvious choice at the time.
Show answer
Model answer: The PRD's error-handling section read "standard error handling applies," pointing at the same banner the booking screen already used. That made sense while the symptom checker was new and untested, and writing three separate messages felt like effort nobody had asked for yet. It stopped making sense the day a low-confidence answer read exactly like a confident one to a worried owner at eleven at night.
Short answer, apply it yourself
5. Pick an AI feature you've used yourself. When it wasn't sure, did it say so, or did it answer anyway and let you find out later? What would a distinct "I'm not confident" message have changed?
Show hint
Look for a moment you trusted an answer you shouldn't have, not a moment it visibly broke.
Show answer
Model answer: "A writing assistant once rewrote a client email with a date it had clearly guessed at, and showed it with the same calm formatting as everything else it wrote correctly. If it had flagged that one line as unconfident, I'd have checked the date before sending instead of after a client pointed it out."
Multiple choice
6. Which of these errors is fine to leave with the plain "Something went wrong, try again" message, according to this answer?
- A. The booking screen failing to save a new appointment time.
- B. The symptom checker answering under its own confidence line.
- C. The symptom checker flagging an emergency pattern.
- D. Any error involving the model, since models are always uncertain.
Show hint
Think about which failure actually costs the user something if the app gets it wrong.
Show answer
A. A booking retry costs five seconds and nothing else. B and C are exactly the failures this answer says need their own state. D overcorrects into needing three states everywhere, which isn't what the answer argues either.
