Artifact critiqueFoundationalDesigning for Uncertainty & Trust / Designing for failure and graceful degradation / #11
Critique an error state that says something went wrong, please try again.
AUDIT a five-word error message is a claim too, and it deserves the same scrutiny as a report
Say we build a meal-planning app whose AI assistant turns a list of ingredients and a family's dietary needs into a week of dinners. Kitchenfold is that app. Consuela Marsh cooks for a household that includes a child with a dairy allergy, and one ordinary week, its error message stopped being honest with her four separate times.
The direct answer
"Something went wrong, please try again" fails an audit because it treats at least four different problems, a timeout, a rate limit, a safety-filter block, and a dead third-party service, as one interchangeable event. Split the message by whether retrying actually helps, and say so plainly, instead of asking the user to guess.
Do this, in order
Never show the same words for causes that need different actions.Why: a message that fits every failure equally well is actually describing none of them.
Say plainly whether retrying will help, hurt, or do nothing.Why: retrying during a rate limit doesn't just fail again, it resets the wait, actively making things worse.
Give a safety-filter block its own message, never the generic one.Why: a block that can only be fixed by rephrasing the request looks, to the user, identical to a server hiccup that fixes itself.
Attach a reference ID to every failure, not just the ones someone complains about.Why: without one, support starts every ticket by re-diagnosing a problem the system already knew the answer to.
Track how often each real cause hides behind the generic message.Why: an aggregate "error rate" says nothing about which of four very different problems is actually growing.
Leave one true one-size-fits-all message for the rare, truly unclassifiable failure.Why: some failures genuinely defy diagnosis, and a plain, honest "we don't know yet" is fine for that sliver.
How to answer this, stage by stage
Nobody is grading whether you can spot that the error message is vague. They're grading whether you can name the specific different things it's hiding.
Stage 1
Treat the error message as a claim, not just copy
Say it like this
"I'm going to audit this the way I'd audit a vendor's benchmark claim. The message is making an implicit claim: 'this is one kind of problem, and trying again is your best move.'"
Why this works
Reframes a UI critique as a trustworthiness question, which is what AUDIT is actually built to test.
Stage 2
Say your structure out loud
Say it like this
"I'll use AUDIT. Ask who wrote it, uncover what's being lumped together, demand specifics on when and where it failed, isolate what's missing, and test it myself."
Why this works
Shows a repeatable way to critique any artifact, not just a one-off complaint about this message.
Stage 3
Uncover what's actually being lumped together
Say it like this
"On Kitchenfold, this exact message covers a backend timeout, a rate limit, a safety-filter block, and a dead price-lookup service, four causes with four different right moves."
Why this works
This is the sharpest move in AUDIT: showing a single label is hiding a real, checkable list underneath it.
Stage 4
Give the one fix
Say it like this
"Split the message by whether retrying helps. A timeout says 'trying again in a moment.' A rate limit says 'wait two minutes, retrying now won't help.' A safety block says exactly what to rephrase."
Why this works
This is the direct answer, stated as a concrete rewrite, not a general complaint about vagueness.
Stage 5
Test it yourself, out loud
Say it like this
"I'd force each of the four failures in a test environment and check: does the message I get actually tell me which one just happened? If two different causes still produce the same text, the audit isn't done."
Why this works
Shows you'd verify the fix rather than assume a rewrite is automatically correct.
Stage 6
Close on the one line
Say it like this
"An error message that fits every failure equally well isn't neutral, it's just wrong four different ways at once. The fix isn't friendlier words, it's telling the truth about which failure this actually is."
Why this works
Restates the direct answer in one breath, ready for a follow-up.
Let's learn
Say we build a meal-planning app that turns a family's ingredients and dietary needs into a week of dinners. When it works, it saves someone about forty minutes a week they used to spend planning by hand.
When it doesn't work, right now, it shows five words: "Something went wrong. Please try again." Every single time, no matter what actually happened underneath.
Four genuinely different problems walk in. One identical, unhelpful sentence walks out.
What's actually behind Kitchenfold's generic error, last month
Four causes, four different right responses, and every single one of them currently gets the exact same five words.
Here's the turn: the vagueness itself was never the real problem. The real problem is that "try again" is actively wrong advice for two of these four causes, and a user has no way to tell which kind of wrong she's about to get.
At its worst, a rate-limited user mashes retry five times, resetting her own cooldown clock each time, and concludes the whole app is broken when the honest answer was simply "wait two minutes."
Only one of these four actually gets better if you just click the button again.
The decision I would take back
Kitchenfold's early engineers wrapped every backend failure in one shared catch block with one generic message, since writing distinct copy for each failure path felt like unneeded work back when the app only had one simple API call to worry about. That made sense at launch. It stopped making sense once the app added an AI assistant, a safety filter, and a third-party price feed, each with genuinely different ways to fail.
What I would leave alone: a truly unclassifiable failure, the rare case that genuinely defies diagnosis even after real investigation, is fine to leave with a plain, honest "we're not sure yet, here's a reference number." That's not the same failure as the four common, nameable ones above.
The lesson: "something went wrong" isn't neutral or safe. It's a specific claim, that the cause is unknown and unknowable, and most of the time that claim is simply false.
Now here is the same thing as a story
The short version above is what you'd say auditing this message in a design review. Read this one for how it wore Consuela down across an actual week.
The error card itself never changes: a plain white box, a single line of grey text, and one blue button that says "Try again." It's the same card, pixel for pixel, every time it appears.
Monday evening, Consuela hit "Generate this week's plan" and got that card. The AI backend had simply timed out under load. She tapped Try again once, and it worked instantly, the plan she wanted, dairy-free swaps and all.
Knowledge spark: why do so many different backend problems produce identical error text?
Many apps wrap every external call, an AI request, a rate limiter, a third-party API, in one shared error handler that catches anything unexpected and shows one fallback message. It's a simple, defensive habit. The cost is that the message inherits none of the specific information the system actually had at the moment of failure.
Tuesday, in a hurry before dinner, she tapped Generate three times in quick succession when the first attempt seemed slow. That tripped a rate limit. The exact same card appeared. She tapped Try again twice more, each tap quietly resetting her own cooldown timer, and gave up for the night having made the wait longer, not shorter.
Three different causes in three days, and the app told her the same thing about all of them: nothing.
Wednesday, she typed a request describing her son's dairy allergy and asked for high-calcium substitutes for a growing child, phrasing that brushed close enough to medical-sounding language to trip the app's safety filter, built to stop the assistant from giving anything resembling medical advice. Same card. She retried four times across ten minutes. It never once helped, because no amount of retrying fixes a request the filter is going to catch every time.
The message never lied about something being wrong. It just never once told her which wrong it actually was.
By Thursday, Consuela stopped opening the meal-planning feature at all and went back to writing the week's dinners on a sticky note, the way she had before Kitchenfold existed.
Same three failures, two very different weeks for Consuela.
With the redesigned error states, Wednesday's message would have read: "This request looks like it needs medical guidance we can't give. Try rephrasing without symptom or health details, for example: 'dairy-free, high-calcium dinner ideas.'" Tuesday's would have read: "You're generating plans quickly. Please wait 2 minutes, retrying now won't speed this up." Run the same week forward: she rephrases once on Wednesday and gets her plan in under a minute, and never once wastes a tap on Tuesday's countdown.
The old card had none of these four. The new one needs all four to actually be honest.
The old message asked Consuela to guess at a problem the system already understood perfectly well. The new one just tells her.
I approved the single shared error handler because writing four separate messages felt like effort spent on an edge case. It took one lost user, and a sticky note back on the fridge, to see that four different edge cases happening every week were never really edge cases at all.
AUDIT, in one screenNot a lecture on error copy. AUDIT is what tells you whether an artifact, even a five-word one, is telling the truth.
A
Ask who wrote it, and why.
An engineer wrapping every failure path in one shared catch block, optimizing for writing less code, not for what the user in front of it needs to know.
Names the incentive that shaped the artifact before judging the artifact itself.
U
Uncover what's being lumped together.
A timeout, a rate limit, a safety-filter block, and a dead third-party service, four causes with four different correct next steps, all producing identical text.
This is the hardest step and the sharpest move: naming the real list hiding behind one label.
D
Demand the specifics: when, and which part, failed.
No timestamp, no reference ID, no named step in the pipeline, just a generic sentence with nothing a support agent could act on.
Without this, every complaint starts from zero instead of from what the system already knew.
I
Isolate what's missing.
No distinction between "retrying helps" and "retrying hurts," and no next step offered besides the one button.
What the message leaves out is more damaging here than anything it actually says.
T
Test it yourself.
Force each of the four known failure types in a test environment and confirm the message you get actually names which one happened.
Verifies the fix instead of assuming a rewrite automatically solved the real problem.
Only the timeout genuinely belongs behind a plain "try again" button. The other three each need their own message.
The recap, one line per letter: ask is naming the engineer's incentive behind the shared catch block, uncover is the four distinct causes hiding under one message, demand is the missing timestamp and reference ID, isolate is the missing retry guidance, and test is forcing each failure type and checking the message actually names it.
And if you want to be sure it really works, try it somewhere elseSame five letters, a fitness coaching app instead of a meal planner. One of its four hidden causes is a safety refusal, not a safety block.
Ampfield is a fitness app whose AI coach generates a weekly training plan from a user's goals and recent activity. Yusuf Danladi uses it to train for a first half-marathon. Ampfield's "Something went wrong, please try again" hides four different causes too: a wearable-data sync failure, an expired subscription check, a server timeout, and a genuine safety refusal, the coach declining to generate a plan because Yusuf logged chest pain during his last run and the system wants him to see a doctor before it suggests more training load.
Auditing it the same way: uncover shows the safety refusal isn't a bug at all, it's a deliberate, responsible decision wearing the exact same broken costume as three genuine errors. Isolate shows the real gap, the refusal never says why, so Yusuf assumed the app was simply malfunctioning and kept retrying a decision the system had actually made on purpose, for a real reason.
Ampfield's fix is the same shape: say what actually happened, instead of letting a deliberate decision look like a crash.
Yusuf's retry attempts before understanding what actually happened, old vs new message
Naming the real cause on the first screen didn't just save clicks, it stopped Yusuf from treating a deliberate safety decision as a bug to work around.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "split the message by whether retrying helps, and give a safety block its own honest text," and stop.
Cost: there's no time to rewrite every error path this sprint. Say so honestly, and start with whichever cause is both common and actively harmful to retry, here, the rate limit.
The model gets better, for real: if the safety filter's false-positive rate genuinely drops, that's still not a reason to fold it back into the generic message, a rarer block is exactly the one a generic message will make look like a random glitch.
Where people run it wrong.
They treat "the error rate" as one number, when it's actually several unrelated numbers wearing the same disguise.
They add a friendlier tone to the same vague message and call it a UX improvement, when the missing information was never about tone.
They wait for support tickets to reveal which cause is most common, instead of instrumenting the four causes separately from day one.
How to use it live. When someone hands you a generic error message to critique, don't start with the wording. Ask how many genuinely different failures are hiding behind it, and whether the advice it gives is actually right for all of them.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits an artifact-critique question?
Tap to flip
ANSWER
AUDIT: ask who wrote it, uncover what's lumped together, demand specifics, isolate what's missing, test it yourself.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Consuela Marsh, a home cook using Kitchenfold to plan dairy-free dinners for her son, who hit the same error message four different ways in one week.
3 · WHAT'S LUMPED TOGETHER
What four causes hide behind the one message?
Tap to flip
ANSWER
A backend timeout, a rate limit, a safety-filter block, and a dead third-party price API, each needing a different response.
4 · WHAT'S MISSING
What does the old message never tell the user?
Tap to flip
ANSWER
Whether retrying will help, hurt, or do nothing at all, plus any reference ID or specific next step.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Wrapping every backend failure in one shared catch block with one generic message, reasonable with a single simple API call, wrong once the app added an AI assistant, a filter, and a third-party feed.
6 · THE NUMBER
Fill in the blank: last month, backend timeouts accounted for ___ percent of the generic error's real causes.
Tap to flip
ANSWER
45 percent, the only one of the four causes where "try again" is actually good advice.
7 · THE REPLAY
Same week, redesigned messages. What changes for Consuela?
Tap to flip
ANSWER
Wednesday's safety block tells her exactly what to rephrase, and she gets her plan in under a minute instead of retrying four times over ten minutes.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what's the safety-related cause there?
Tap to flip
ANSWER
Ampfield, a fitness coaching app. The hidden cause is a deliberate safety refusal, the coach declining to generate a plan after Yusuf logged chest pain.
Check yourself Score: 0 / 0
True or false
1. True or false: for all four causes behind Kitchenfold's generic error, retrying is the correct thing to do.
True
False
Show hint
Look at the quadrant diagram.
Show answer
False. Only the timeout genuinely benefits from retrying. Retrying during a rate limit actively makes things worse, and it never helps against a safety block.
Multiple choice
2. What is the sharpest move in AUDIT's approach to this artifact?
A. Making the error message sound friendlier.
B. Uncovering the distinct causes a single generic message is lumping together.
C. Adding an animation to the error card.
D. Removing the retry button entirely.
Show hint
Look at the Uncover step.
Show answer
B. The real problem was never tone. It was that one label was hiding several genuinely different, checkable causes.
Fill in the blank
3. Fill in the blank: with the old generic message, Yusuf on Ampfield retried ___ times before understanding it was a deliberate safety refusal.
Show hint
Look at the line chart in Section 4.
Show answer
6 times. With the redesigned message naming the refusal directly, that number dropped to 0.
Short answer, apply it yourself
4. Think of a generic error message you've seen recently. Could you tell, from the message alone, whether trying again would actually help?
Show hint
Ask whether the message ever distinguished a temporary glitch from something that needed a different action from you.
Show answer
Model answer: Most people can recall retrying something pointlessly, exactly the gap this audit targets.
Short answer, where it wouldn't matter
5. Name a kind of failure where a single generic error message is still an honest, reasonable choice.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: A truly unclassifiable failure that genuinely defies diagnosis. A plain "we're not sure yet, here's a reference number" is honest there, since there's nothing more specific to say.
Short answer, the number question
6. If safety-filter blocks made up 40 percent of Kitchenfold's errors instead of 15 percent, would fixing the message for that cause become more or less urgent, and why?
Show hint
Look at the bar chart in Section 1.
Show answer
Model answer: More urgent. A cause that never benefits from retrying gets worse to leave generic the more often it happens, since more users would be stuck with advice that can never work for them.
Before you close the answer
Why this works
Tests whether you'll audit a UI artifact with the same rigor you'd bring to a report or a claim, or whether you'll treat "just make the copy nicer" as good enough.
Follow-up traps
"Isn't writing four separate messages just more engineering work for a minor polish item?" Response: no, it's core reliability communication. Two of the four causes actively get worse if users follow the generic message's implied advice to retry.
"What if the underlying failure genuinely can't be classified in time?" Response: that's exactly what the one honest fallback message is for, an explicit "we don't know yet" is a true statement, unlike a message that implies a knowable answer it isn't giving.
If pressed
Kitchenfold's redesigned error handling tags each failure with the specific pipeline stage that raised it, the AI call, the safety filter, the rate limiter, or the price API, so the message text is generated from a real, named cause rather than written once and reused everywhere.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
