CaseIntermediateDesigning for Uncertainty & Trust / Designing for failure and graceful degradation / #14
What should a rate-limited user see, and how do you avoid making it feel punitive?
PICK the cost that's cheap to absorb versus the cost nobody files a complaint about
Keyhatch helps renters draft personalized notes to landlords with AI. Here is what happened when Yasmin Torbert hit her daily free limit eight days before her lease ended, and the design that would have kept her searching instead of walking away.
The direct answer
Show the exact count left, the exact reset time, and a plain reason why the limit exists, and hand over a manual fallback she can keep using right now, free. Never show a bare "rate limit exceeded" wall with no path forward. The limit is the cheap decision; a dead end is the expensive one.
Do this, in order
Pair the limit with a manual fallback she can use immediately.Why: a limit with no path forward reads as a punishment, one with a path forward reads as a pause.
Show the exact reset time, not a vague "try again later."Why: an unknown wait feels endless. A countdown feels like a plan she can work around.
Say plainly why the limit exists, in one line.Why: "to keep this free for everyone" reads as a shared reason. Silence reads as a personal rejection.
Never reuse the same screen built to stop bots for a real, rushed renter.Why: the two situations have nothing in common, and the bot-facing tone is cold on purpose.
Watch for real bot abuse before tightening the limit further.Why: a stricter cap punishes the honest renters first, since bots barely notice a cap at all.
How to answer this, stage by stage
Nobody is grading whether you'd rate limit at all. They're grading whether you can name which side of the cost you're actually protecting.
Stage 1
Scope it to one product and one moment
Say it like this
"I'll use Keyhatch, an AI tool that helps renters draft personalized notes to landlords, and the day Yasmin Torbert hit her daily limit eight days before her lease ended."
Why this works
Gives you a real screen and a real stake to design around, not a policy in the abstract.
Stage 2
Say your structure out loud
Say it like this
"I'll use PICK. Position, impact, cost asymmetry, kill criteria."
Why this works
Signals you're about to commit to a side, not list pros and cons forever.
Stage 3
State your position before any reasoning
Say it like this
"Show what's left, when it resets, why it exists, and give her a manual fallback right there. Never a dead end."
Why this works
Answers the actual question in one breath, before the interviewer wonders if you're hedging.
Stage 4
Name who feels each kind of error
Say it like this
"If we don't limit at all, Keyhatch eats the inference cost on its busiest days. If we limit it harshly with no explanation, Yasmin feels shut out during the one week it mattered most."
Why this works
Puts a real person on both sides of the tradeoff, not just a cost line and a vague "user experience."
Stage 5
Name the asymmetry and pick a side
Say it like this
"A little extra cost is cheap and visible, we can budget for it. A renter's trust breaking silently, right when she needed the tool most, is hidden and expensive. Optimize against the second one."
Why this works
This is the heart of PICK: naming which error is actually the dangerous one, in the person's terms.
Stage 6
Give the kill criteria
Say it like this
"If we start seeing the free tier hit by bots and scrapers instead of real renters, that changes the math, and I'd tighten it even though real renters would feel some of that too."
Why this works
Shows the position isn't stubborn, there's a real condition that would flip it.
Stage 7
Close on the replay
Say it like this
"Run the same day forward with the redesigned screen, and Yasmin finishes her most urgent application in twenty minutes using the free template, instead of closing the app for good."
Why this works
Ends on something countable, not a claim that the new screen just "feels nicer."
Let's learn
Keyhatch is an app that helps renters write personalized notes to landlords, using AI to turn a few facts into something a landlord will actually read.
Before any limit, renters like Yasmin used Keyhatch freely, drafting a note in under a minute each time, versus roughly fifteen minutes writing one by hand.
Once usage climbed past what the free tier could sustain, Keyhatch shipped a daily cap of six AI drafts per renter, reusing the same rate-limit screen built earlier to stop bots scraping listings: a plain red banner reading "Rate limit exceeded. Try again later," no count, no reset time, no reason given.
The redesigned path. The old one stopped at box one.
Here's the turn: the limit itself was never the problem, six free notes a day is a reasonable, sustainable number. What actually hurt was a screen with no explanation and nowhere to go, showing up on exactly the day a renter needed the tool most.
Share of renters who opened Keyhatch again the next day, by how the limit screen was shown
Same six-draft limit, same renters. Only the screen changed, and it more than tripled who came back.
At its worst, a renter facing a real deadline hits a wall with no explanation, assumes the app is broken or done with her, and quietly switches to a competitor for the one listing that actually mattered.
One of these costs shows up on a spreadsheet. The other one shows up nowhere, until renters just stop coming back.
The decision I would take back
Keyhatch wired the renter-facing daily cap to the exact same rate-limit component built to stop bots scraping listings, one generic "Rate limit exceeded" screen for both. That made sense when engineering only had one rate-limiter to build and bot traffic was the only thing hitting it. It stopped making sense the moment real renters, not bots, started hitting the same wall during the week they needed the tool most.
What I would leave alone: the six-draft number itself doesn't need to change. It's already generous enough for how renters actually use the tool; the failure was entirely in what the screen said, not in where the line was drawn.
The lesson: a limit and a punishment are not the same thing, but a screen with no explanation and no way forward makes them feel identical. The number was never what hurt. The silence around it was.
Now here is the same thing as a story
The short version above is what you'd say defending this redesign to Keyhatch's product council. Read this one for how the week actually went.
Yasmin Torbert is good at noticing exactly which detail a landlord actually cares about, a flexible move-in date, a quiet reference from a current employer, the kind of thing that gets a note actually read instead of skimmed. She'd written these by hand for years before Keyhatch, about fifteen minutes each, and she knew which ones worked.
Her first week on Keyhatch was good. She applied to twelve places using three or four drafts a day, well under the limit, never noticing it was even there.
Knowledge spark: why does a free AI feature need a limit at all?
Every AI-generated draft costs the company real money to produce, a few cents each, but it adds up fast across thousands of renters. A limit isn't about punishing anyone, it's the line between a free feature the company can actually afford to keep offering and one it eventually has to shut down for everyone.
Eight days before her lease ended, with three strong listings closing their applications within forty-eight hours, Yasmin started drafting faster, six, seven, eight notes a day, chasing every option she could still reach in time.
The limit shipped two weeks before Yasmin's own usage climbed high enough to actually meet it.
On the morning her favorite listing's application window closed in twenty-four hours, she opened Keyhatch to draft her strongest note yet, and hit a flat red banner: "Rate limit exceeded. Try again later." No count. No reset time. No reason.
The limit did not cost Yasmin six drafts. It cost her the one she needed most, on the one morning it mattered.
She closed the app, assumed it had broken or was done with her for the day, and switched to a free-text competitor app for that listing, losing the personalization Keyhatch was actually good at, on her most important application of the week.
This is what the redesigned screen shows Yasmin. The old one showed none of it.
With the redesigned screen, Keyhatch tells her: "6 of 6 free notes used today. Resets in 6h 40m. Want to keep going right now? Here's a plain template, free, anytime, just fill in the details yourself." Run the same morning forward: she takes the template, finishes her application to the closing listing in twenty minutes, and opens Keyhatch normally again the next day.
All four of these fit on one screen. None of them cost anything extra to build.
The old screen asked Yasmin to just wait and wonder. The new one gave her something to do while she waited.
I let the limit ship on the same screen we'd built for bots, because it was already there and the deadline was tight. It took watching a renter with a real lease deadline hit that same wall to see that "already built" and "actually fits" were never the same thing.
PICK, mapped onto the limit screenNot a lecture on being nice about limits. PICK is what tells you which cost you're actually protecting against.
P
Position. Your pick, before any reasoning.
Show the count left, the reset time, the reason, and a manual fallback. Never a bare wall.
This is the hardest step and the answer to the question: a real commitment, not a menu of options.
I
Impact. Who feels each error, and in what units.
No limit: the company absorbs extra inference cost on busy days. Harsh limit: the renter loses momentum during her highest-stakes week.
Names a real person on both sides, not just a cost line versus a vague complaint.
C
Cost asymmetry. Which one is hidden and expensive.
Extra cost is cheap, visible, and budgetable. A renter's trust breaking silently at the worst possible week is hidden and much more expensive.
The heart of the framework: naming which error you're actually optimizing against.
K
Kill criteria. What would flip this pick.
Clear bot or scraper abuse of the free tier, or cost per active renter crossing a sustainable ceiling, would justify a stricter limit.
Shows the position is a real judgment call, not stubbornness.
The recap, one line per letter: position is a generous, explained limit with a fallback, impact names both the company's cost and the renter's momentum, cost asymmetry says the renter's silent trust break is the one to protect against, and kill criteria is bot abuse or a broken cost ceiling.
The same asymmetry that opened Section 1 is the thing PICK's C step is actually naming.
Monthly AI cost per active free renter, as the daily draft limit rises
Cost crosses the sustainable ceiling around 7 to 8 drafts a day. The current limit of 6 sits safely under it, with real room before this pick needs to flip.
And if you want to be sure it really works, try it somewhere elseSame four letters, an insurance claims portal instead of a rental app. A different kind of urgency, the same asymmetry.
Cobblestone Mutual runs SwiftClaim, an AI tool that helps policyholders draft storm-damage claim descriptions. Godfrey Okonkwo tried to file a claim through it three days after a regional storm, right when everyone else in his zip code was filing too. Mapped onto PICK: position is showing his place in a fast manual queue instead of a bare error the moment AI-assisted drafting is full, since a disaster surge is exactly when policyholders most need to feel the company is still there.
The impact split here is sharper than Keyhatch's: no limit at all risks the AI drafting overwhelmed claims sloppily under load, missing details that matter for the payout. A harsh limit with no explanation risks a policyholder who just lost part of a roof feeling abandoned by their own insurer during a disaster. The cost asymmetry: a slower manual queue is visible and survivable, a policyholder concluding "my insurer vanished when the storm hit" is the kind of story that spreads and doesn't come back. The kill criteria here is different too: a spike in claims that look fraud-shaped, filed suspiciously fast right after the storm, would justify slowing the AI-assist down regardless of genuine demand.
Same four parts as Keyhatch's screen. A queue position instead of a fallback template, because the underlying claim still has to get written.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "show the count, the reset time, the reason, and a fallback, and never a bare wall," and stop.
Cost: there's no time to build a real fallback template this sprint. Say so honestly, and ship the reset time and the reason first, since those two alone already change how the wall feels.
The model gets better, for real: if AI drafting gets cheap enough that a limit is barely needed, that's still not a reason to remove the explanation and fallback, they cost nothing and keep working even after the limit itself relaxes.
Where people run it wrong.
They reuse a rate-limit screen built for bots and never notice a real, rushed person is now the one reading it.
They treat "add a friendlier sentence" as the whole fix, without actually giving the person something to do next.
They assume anger about a limit will show up as a complaint, when most people who feel shut out just leave quietly instead.
How to use it live. When someone hands you a rate-limited screen and asks you to make it feel less punitive, ask yourself one question before rewriting a single word: does this screen give the person something to do right now, or just something to wait for? Rewrite until the answer is the first one.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits a "rate limit, hard call or soft" question?
Tap to flip
ANSWER
PICK: position, impact, cost asymmetry, kill criteria. Commit to a side, then show which error is actually hidden and expensive.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Yasmin Torbert, a renter who wrote her own landlord notes by hand for years, applying to twelve places using Keyhatch in her first week.
3 · THE POSITION
What's the actual pick, in one sentence?
Tap to flip
ANSWER
Show the count left, the reset time, the reason, and a manual fallback. Never a bare "rate limit exceeded" wall.
4 · THE ASYMMETRY
Which cost is cheap and which is hidden here?
Tap to flip
ANSWER
Extra inference cost is cheap and visible, easy to budget for. A renter's trust breaking silently during her highest-stakes week is hidden and far more expensive.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Reusing the same rate-limit screen built to stop bots for real, rushed renters hitting a legitimate daily cap.
6 · THE NUMBER
Fill in the blank: renters shown the bare wall returned the next day ___ percent of the time. Renters shown the redesigned screen returned ___ percent of the time.
Tap to flip
ANSWER
22 percent versus 68 percent. Same six-draft limit, more than triple the return rate, just from what the screen said.
7 · THE REPLAY
Same morning, redesigned screen. What changes?
Tap to flip
ANSWER
Yasmin takes the free manual template, finishes her most urgent application in twenty minutes, and opens Keyhatch again normally the next day.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what's the position there?
Tap to flip
ANSWER
SwiftClaim, Cobblestone Mutual's AI claims tool. The position is showing a fast manual queue position during a disaster surge, instead of a bare limit wall, since abandonment there risks far more than a delayed draft.
Check yourself Score: 0 / 0
True or false
1. True or false: the fix for Yasmin's bad experience was raising the daily draft limit above six.
True
False
Show hint
Look at "what I would leave alone."
Show answer
False. The number six was already fine. The failure was a screen with no explanation and no fallback, not the size of the limit.
Fill in the blank
2. Fill in the blank: renters shown a bare rate-limit wall returned the next day ___ percent of the time, versus 68 percent for renters shown the redesigned screen.
Show hint
Look at the grouped bar chart.
Show answer
22 percent. More than a three-fold gap, from the screen alone, with the exact same underlying limit.
Multiple choice
3. Which error does PICK's cost asymmetry say Keyhatch should optimize against?
A. The company absorbing a little extra inference cost on busy days.
B. A renter's trust breaking silently during her highest-stakes week.
C. The AI drafting notes slightly slower than usual.
D. Landlords receiving too many personalized notes.
Show hint
Look at the C step, cost asymmetry.
Show answer
B. It's hidden, since she never files a complaint, and expensive, since she may not come back at all.
Short answer, apply it yourself
4. Think of a free tool you use with some kind of daily or monthly cap. What does it show you when you hit it, and does it give you something to do right now?
Show hint
Ask whether it shows a count, a reset time, and a next step, or just an error.
Show answer
Model answer: Most caps show only an error or an upgrade prompt, skipping the count, the reset time, and any free path forward, exactly the gap this answer redesigns around.
Short answer, name the reversal
5. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at the key point block titled "The decision I would take back."
Show answer
Model answer: Reusing the bot-facing rate-limit screen for real renters. It made sense when only bots were hitting the limit and engineering had one rate-limiter built already.
Short answer, where it wouldn't matter
6. If the daily draft limit had to drop from six to two tomorrow, would this same redesign still be enough on its own? Why or why not?
Show hint
Look at the kill criteria step and the cost chart.
Show answer
Model answer: Not fully. A count, reset time, reason, and fallback still help, but a limit that low would need a real explanation of why, and probably a stronger fallback, since two drafts barely covers one urgent day.
Before you close the answer
Why this works
Tests whether you'll commit to protecting the hidden, expensive cost, a renter's trust breaking silently, rather than the cheap, visible one, a little extra server bill. Most candidates hedge or focus only on the number.
Follow-up traps
"Isn't a generous limit just going to get abused?" Response: watch for real bot or scraper signatures, not just heavy real usage, and tighten only when that evidence shows up, per the kill criteria.
"Why not just remove the limit and eat the cost?" Response: unlimited free usage eventually forces killing the feature for everyone, which is worse for every renter than a generous, well-explained cap.
If pressed
Keyhatch's redesigned reset countdown is computed from the renter's own local time zone and rounds down to the nearest five minutes, so the number on screen never quietly drifts wrong as a session sits open past midnight.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
