Artifact critiqueIntermediateDesigning for Uncertainty & Trust / Designing for failure and graceful degradation / #6

What error message would you write for a model timeout, and what would you avoid saying?

AUDIT judge the message actually on screen before you write a better one

SkyQuery is an AI flight search assistant that lets travelers describe a trip in plain language instead of filling in a rigid form. Franklin Osei writes the product's user-facing copy. Here is the exact message SkyQuery showed when its search model timed out, and what a real audit of that message actually found.

The artifact under review, verbatim, as shipped Error: upstream provider timeout after 12000ms. retry_code=504.
The direct answer
Say what happened in plain words, say whether anything was charged, and give one thing to do next. "Your search took too long and timed out. You weren't charged. Try again, or narrow your dates." Never show an internal error code, a millisecond count, or the word "provider" to a traveler, and never blame the search itself for being too hard.
Do this, in order
  1. State plainly what happened, in words a traveler would use.Why: "upstream provider timeout" describes the system's internals, not the traveler's actual experience of a slow search.
  2. Say whether they were charged, every single time, even when the answer is no.Why: the scariest part of a failed action is never knowing what state you're actually in.
  3. Give exactly one next step.Why: a message with no action is a dead end dressed up as information.
  4. Never show an internal error code, a timestamp, or the word "provider."Why: none of it is actionable to a traveler, and all of it reads as evidence that something is badly broken.
  5. Never blame the traveler's own request for being too complex.Why: it was a genuinely reasonable search. The system's limits aren't the traveler's fault.
  6. Leave the simple, fast-path searches alone.Why: this message only ever appears on the rare slow query. It shouldn't change anything about the common, quick one.

How to answer this, stage by stage

Nobody is grading whether you can write a nicer sentence. They're grading whether you can judge the one that's already live before you touch it.

Stage 1
Scope it to one real message
Say it like this
"I'll audit the actual message SkyQuery shipped for a search timeout, 'Error: upstream provider timeout after 12000ms, retry_code=504,' before I write a replacement."
Why this works
Grounds the critique in a real artifact instead of an abstract idea of "a bad error message."
Stage 2
Say your structure out loud
Say it like this
"I'll use AUDIT. Ask who it serves, uncover what actually failed, demand a next step, isolate what's missing, and test it myself as a real traveler would read it."
Why this works
Shows a repeatable way to judge any shipped message, not just an opinion about this one.
Stage 3
Ask who the current message actually serves
Say it like this
"This message serves whoever's reading server logs. It names a retry code and a millisecond count, exactly what an engineer debugging the outage would want, and nothing a traveler standing at an airport gate would."
Why this works
Names the real audience mismatch instead of just calling the message "bad."
Stage 4
Isolate what's missing
Say it like this
"No next step, no reassurance about whether anything was charged, and no plain description of what actually happened. Just an internal string, verbatim from a server log."
Why this works
Turns a vague "this feels wrong" reaction into three specific, fixable gaps.
Stage 5
Give the rewrite, the one decision
Say it like this
"'Your search took too long and timed out. You weren't charged. Try again, or narrow your dates.' Plain words, a reassurance, one action."
Why this works
This is the direct answer, an actual message someone could ship today, not a description of good qualities.
Stage 6
Close on what to never say
Say it like this
"Never show an internal code or a provider name. Never blame the traveler's request for being complex. And never promise 'this won't happen again' if you can't actually back that up."
Why this works
Answers the second half of the question directly, not just by implication from the rewrite.

Let's learn

SkyQuery is a flight search tool that reads a loose request, "cheapest way to Lisbon in October, no red-eyes," and searches across many partner airlines and fare combinations to answer it, instead of forcing a traveler into a rigid origin, destination, and date form.

Hand sketched flow diagram titled How the raw message reached the screen. Five boxes: Model call times out, Exception thrown, Dev logging string, No translation layer highlighted, Shown to traveler.
The message was never written for a traveler. It was written for a log file, and nothing ever stood between the two.

Before SkyQuery's flexible search, travelers used a basic form with fixed fields, slower to fill out, but it always returned a result list in under two seconds, since there was no open-ended question for a model to think through.

Here's the turn: the flexibility itself was never the problem. As more travelers typed in loose, multi-city, flexible-date questions, more searches ran long enough to time out, and the message that greeted them when that happened had never been written for a traveler at all. It was the exact string an engineer would see in a server log, sent straight to the screen with nothing translated in between.

Weekly support tickets mentioning the timeout message, before and after the rewrite
60 30 0 message rewritten Week 1, 8 tickets Week 6, 61 tickets Week 8, 9 tickets
The rewrite alone, no change to how often the search actually times out, cut ticket volume by more than 80 percent within two weeks.

At its worst, a traveler reads "retry_code=504," assumes the worst, closes the tab, and books directly on an airline's own site instead, at a higher fare, simply because a string built for a log file made a normal timeout look like a serious failure.

Hand sketched comparison titled The message under review. Left, a grey box icon labeled Old message, caption engineering's error verbatim. Right, a blue document icon labeled New message, caption the traveler's next move.
Same failure, same server, same twelve seconds. Only what the traveler was told about it changed.
The decision I would take back When SkyQuery's search feature first shipped, nobody built a translation layer between what the model's error handling produced internally and what a traveler would actually see. Passing the raw string straight through felt like a reasonable shortcut for a first release, since timeouts were rare and the team was watching logs closely anyway. It stopped making sense once loose, flexible searches became common enough that a real traveler, not an engineer, was the one reading that string most weeks.

What I would leave alone: the fast, simple searches that never time out don't need any of this. This message only exists for the rare slow query, and nothing here should touch the common, quick path.

The lesson: an error message is the one piece of copy a product writes for someone who is already anxious. It's the worst possible place to leave in a sentence meant for a different reader.

Now here is the same thing as a story

The short version above is what you'd say defending this rewrite to SkyQuery's product council. Read this one for how the message survived unnoticed for as long as it did.

Franklin writes most of the words a traveler sees on SkyQuery, confirmation screens, empty states, the little reassurances that show up when a search is running. For a long time, the timeout message wasn't on anyone's list to review, since it almost never fired.

Knowledge spark: why does a raw error string end up shipped to real users? When a feature is new, engineers often wire up basic error handling fast, printing whatever the underlying exception says, purely so they can debug it themselves while the feature is still rough. That raw string is meant to be a placeholder, swapped out for real user-facing copy before most people ever see it. The risk is that "before most people see it" quietly becomes "after thousands of people already have," especially once a feature scales past what the team originally expected.

As SkyQuery's flexible search grew popular, more loose, multi-city, flexible-date questions came in, and more of them ran long enough to hit the model's timeout. Nobody had been tracking that number on its own, since it lived buried inside a broader "search success rate" that still looked mostly fine.

Hand sketched timeline titled The week complaints spiked. Five milestones: Loose date search ships week 1, Timeouts start climbing week 3, Raw error hits users week 4 highlighted, New hire asks why week 6, Message rewritten week 7.
Three weeks between the raw message quietly reaching real travelers and anyone on the team asking why it said what it said.

A new hire on the support team, going through a backlog of tickets in her second week, asked in a channel: "why does our timeout error say 'retry_code=504'? What is a traveler supposed to do with that?" Nobody had a ready answer. It had simply never been anyone's job to notice.

The message never lied. It just spoke only to the one reader who'd never actually see it on a real trip.

Franklin pulled the actual message and sat with it the way a traveler would, mid-search, phone in hand, trying to book a trip before a fare changed. Read cold, it gave no next step, no word on whether a charge had gone through, and a string of characters that looked, to anyone outside engineering, like proof something had badly broken.

Hand sketched icon list titled What the audit found missing. Three items: a question mark box icon labeled No next step for the traveler, a scale icon labeled No word on whether they were charged, a gauge icon labeled Internal codes exposed unexplained.
All three gaps existed in the exact same twelve words. None of them required a single change to how the search itself works.

With the redesigned message, a timeout now reads: "Your search took too long and timed out. You weren't charged. Try again, or narrow your dates to speed it up." Run the same search forward: the traveler who once closed the tab in frustration now taps "try again," narrows her dates, and gets a result in under ten seconds.

Hand sketched labeled parts diagram titled Anatomy of a message that survives. A document icon at the center labeled Good timeout message, with four callouts around it: what happened plain, were you charged, one next step, no blame.
The old message answered none of these four questions. It answered a question nobody standing at a gate was actually asking.

The old message told a traveler exactly what an engineer needed to know. The new one tells her exactly what she needs to know instead.

We let that raw string ship because it was never meant to last, just a placeholder for a feature still finding its feet. It took a new hire's plain question to notice that "temporary" and "shipped to thousands of travelers a week" had quietly become the same thing.

AUDIT, aimed at a single error messageNot a style guide for cheerful copy. AUDIT is what forces you to judge the exact words already on screen before writing new ones.

A
Ask who it serves.
The raw message serves whoever reads server logs. A traveler mid-search gets nothing written for them at all.
Names the real audience mismatch instead of a vague sense that the copy is unfriendly.
U
Uncover what actually failed.
A model call ran past its own time budget, not a payment failure, not a lost booking. The message never says even that much in words a traveler would recognize.
Separates what genuinely happened from what the message chose to say, or leave out.
D
Demand a next step.
"Try again, or narrow your dates" gives the traveler something to do in the next ten seconds. The old message gave nowhere to go at all.
The hardest step and the real test of any error message: a message with no action is just a status report nobody asked for.
I
Isolate what's missing.
No reassurance about charges, no plain description, an internal code with no explanation, all three gone from the same twelve words.
Turns a vague complaint into three specific, fixable gaps a rewrite has to close.
T
Test it yourself.
Read cold, mid-search, phone in hand, the old message reads like proof of a serious failure. That's the actual bar a rewrite has to clear.
Replaces a guess about how the message lands with an honest read as the person who'll actually see it.
Hand sketched quadrant titled Which errors need which message. Axes how sure we are what failed from unsure to certain, and how much the traveler can do about it from nothing to a lot. Search timeout sits high on both axes. Card declined sits high on both axes. Rare internal bug sits low on both axes.
A timeout is exactly the kind of error where the traveler can genuinely act. That's what earns it a specific, confident message instead of a vague one.

The recap, one line per letter: ask who it serves finds the audience mismatch, uncover what actually failed separates the real cause from the code, demand a next step gives the traveler something to do, isolate what's missing names the three gaps plainly, and test it yourself checks the rewrite against a real, anxious reading.

And if you want to be sure it really works, try it somewhere elseSame five letters, a hotel's overbooking notice instead of a flight search timeout. A different failure, the same audit.

Cordova Stays runs a booking platform where an overbooking-prevention model occasionally has to tell a guest their exact room type sold out after they'd already booked it. Nadia Fitch handles guest relations for the chain. Mapped onto AUDIT: the old message read "Inventory conflict: SKU unavailable, booking reference invalidated," written for the reservations system, not the guest holding a confirmation email.

Asking who it serves finds the same mismatch: an internal inventory term, not a guest's actual situation. Uncovering what failed means naming that a genuine double-booking happened, not implying the guest did anything wrong. Demanding a next step means the message has to offer a specific alternative room or a full refund path, immediately, not just an apology. The redesigned message reads: "We're sorry, your exact room type sold out due to a booking conflict on our end. We've held you an upgraded room at no extra cost, or you can cancel for a full refund." Testing it cold, as a guest who already packed a bag, is what caught that the first draft still buried the refund option two screens deep.

Hand sketched labeled parts diagram titled Anatomy of a message that survives, reused here for a hotel overbooking notice. Center document icon labeled Good timeout message, with what happened plain, were you charged, one next step, and no blame around it, relabeled for an overbooking apology.
Swap "were you charged" for "here's your room or your refund," and the same four parts still do the same job.
Guests who called support after an overbooking notice, old versus rewritten message
80% 40 0 Old message, 71% New message, 19%
Same double-booking rate. Naming the fix immediately, instead of an inventory term, cut support calls by more than half.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "plain words, a reassurance, one next step, no internal codes, no blame," and stop.
Cost: there's no engineering time to build a different message for every possible internal error code this quarter. Say so honestly, and start with the errors travelers actually hit most, timeouts and payment failures, before covering every rare internal case.
The model gets better, for real: if SkyQuery's search speed genuinely improves and timeouts become rare, that's still not a reason to let the raw string creep back in, the rarer the error, the more surprising and alarming an unexplained code will read.

Where people run it wrong.
They treat error copy as an engineering afterthought instead of the one moment a product writer should care about most.
They add a friendlier tone to the same unhelpful content, instead of actually giving the person a next step.
They promise certainty they don't have, "this will never happen again," which reads as hollow the second time it does.

How to use it live. When someone hands you an error message and asks if it's good, read it once as the exact person who'll see it, mid-task, a little anxious, and ask whether it tells them what happened, whether they're safe, and what to do next. If it's missing any of the three, it isn't ready.

Flashcards (tap any card to flip it)

1 · THE FRAMEWORK
What framework fits judging a shipped artifact like an error message?
Tap to flip
ANSWER
AUDIT: ask who it serves, uncover what failed, demand a next step, isolate what's missing, test it yourself as the real reader would.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Franklin Osei, the product writer at SkyQuery who audited the timeout message after a new hire asked what it actually meant.
3 · THE ARTIFACT
What was the actual message under review?
Tap to flip
ANSWER
"Error: upstream provider timeout after 12000ms. retry_code=504." A raw engineering string shipped straight to travelers.
4 · THE REWRITE
What's the redesigned message?
Tap to flip
ANSWER
"Your search took too long and timed out. You weren't charged. Try again, or narrow your dates to speed it up."
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Never building a translation layer between the model's internal error handling and what a traveler actually sees, since timeouts were rare when the feature first shipped.
6 · THE NUMBER
Fill in the blank: weekly support tickets mentioning the timeout error peaked at ___ before the message was rewritten.
Tap to flip
ANSWER
61 tickets, up from 8 in week one. It fell to 9 within two weeks of the rewrite, with no change to the search itself.
7 · WHAT TO AVOID
Name two things this answer says to never put in a timeout message.
Tap to flip
ANSWER
An internal error code or provider name, and any blame placed on the traveler's own request for being complex.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what was the old message there?
Tap to flip
ANSWER
Cordova Stays' hotel booking platform. The old message read "Inventory conflict: SKU unavailable, booking reference invalidated" for a guest whose room had been overbooked.

Check yourself Score: 0 / 0

True or false
1. True or false: the redesigned timeout message includes the retry code so travelers can reference it if they contact support.
  • True
  • False
Show hint
Look at the direct answer and the priority list.
Show answer
False. The redesigned message deliberately drops the internal code entirely, since it isn't actionable to a traveler and reads as evidence something is badly broken.
Multiple choice
2. Why does the audit say the old message "serves" an engineer rather than a traveler?
  • A. Engineers were the only ones who ever saw the message in testing.
  • B. It states a millisecond count and a retry code, information useful for debugging, and gives no next step a traveler could act on.
  • C. The message was written in a technical font.
  • D. SkyQuery's legal team required the exact wording.
Show hint
Look at the Ask who it serves step.
Show answer
B. The content itself, a retry code and a timing detail, is exactly what a log reader wants and a traveler cannot use.
Fill in the blank
3. Fill in the blank: it took about ___ weeks between the raw message reaching real travelers and a new hire asking why it said what it said.
Show hint
Look at the timeline diagram.
Show answer
Three weeks. The message reached users in week 4 and the question that triggered the audit came in week 6.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when SkyQuery's search feature first shipped?
Show hint
Look at "the decision I would take back."
Show answer
Model answer: Never building a translation layer for the raw error string. It made sense when timeouts were rare and mostly seen by the engineering team itself.
Short answer, apply it yourself
5. Think of a confusing error message you've seen recently, in any app. Read it now as if you were a stranger seeing it for the first time. What's missing: what happened, whether you're safe, or what to do next?
Show hint
Check the message against all three of AUDIT's isolate-what's-missing categories.
Show answer
Model answer: Most real error messages are missing at least one of the three, most often the next step, which is exactly the gap this answer's rewrite closes first.
Short answer, where it wouldn't matter
6. Name a part of SkyQuery where this timeout-message audit genuinely doesn't need to apply.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: The fast, simple searches that return in under two seconds. They never time out, so this message never appears for them at all.
Before you close the answer
Why this works
Tests whether you can judge a real, already-shipped piece of copy on its own terms, instead of jumping straight to writing something that merely sounds nicer.
Follow-up traps
"Isn't the error code useful for support staff, even if not for the traveler?" Response: yes, but it should live in a logging system support can query, not in the message a traveler reads. The two audiences need two different surfaces, not one message trying to serve both.

"What if you genuinely don't know why the search failed?" Response: say that honestly, "we couldn't complete your search," rather than inventing a plausible-sounding but wrong explanation. An honest unknown beats a confident guess that turns out false.
If pressed
SkyQuery's redesigned error system logs the full internal string, retry code and all, to an internal dashboard support can search by timestamp, so nothing useful to engineering was actually lost by removing it from the traveler-facing message.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more