CaseAdvancedDesigning for Uncertainty & Trust / Trust, transparency and explainability in UX / #8
Describe how you would design for a user who wants to audit an AI decision.
TRACE the product is Havenwell, a rental marketplace, and its tenant-screening assistant
Havenwell is a rental marketplace. Its screening assistant reviews an applicant's credit, income, and rental history, then flags a share of applications for a property manager's closer look. Tobias Reyes applied for an apartment through Havenwell and got flagged with no explanation at all.
The direct answer
Design the audit as a trace, not a reveal. Show which specific factors the decision used and exactly when each one entered the file. Rule out a plain data error, like a stale credit pull, before showing any model judgment at all. Name the handful of real reasons instead of one vague flag. Then give the applicant one clear, checkable action that would change the outcome if he's right that something's wrong.
Do this, in order
Rule out stale or wrong underlying data before showing any model judgment.Why: a data problem looks exactly like a real risk factor until someone checks the date on it.
Name the handful of real cause candidates, not one vague "additional review" message.Why: a person can't act on a flag they can't even name.
Let the applicant slice into any single factor and see its exact weight.Why: a decision made of five factors isn't auditable if it's only ever shown as one number.
Give one specific, checkable action that would flip the outcome, if it's true.Why: an explanation with nothing to do next is just a longer way of saying no.
Keep a permanent, timestamped record of which inputs drove the decision.Why: without one, nobody, not even Havenwell, can reconstruct a disputed decision months later.
How to answer this, stage by stage
Nobody's grading whether you know audit trails are good practice. They're grading whether you can build the actual screen a real applicant uses to trace a flag back to its cause.
Stage 1
Scope it to one real decision
Say it like this
"I'll answer this for Havenwell's tenant-screening flag, and the specific applicant, Tobias, who got one with no explanation."
Why this works
Turns "design an audit feature" into one concrete decision you can actually trace.
Stage 2
Say your structure out loud
Say it like this
"I'll use TRACE. Timeline, recut by factor, assume nothing about data quality, name the cause candidates, then give one evidence test."
Why this works
TRACE was built for diagnosing what changed. Auditing one person's decision is the exact same shape of problem, aimed at one file instead of a whole metric.
Stage 3
Show the timeline
Say it like this
"The screen opens with a timeline: application submitted day one, credit pulled that same day from a report that was already three weeks old, flagged day two."
Why this works
Puts the exact sequence in front of the applicant, instead of a single frozen result.
Stage 4
Assume nothing about the data first
Say it like this
"Before showing any judgment about risk, the screen checks: is the underlying data even current? Tobias's credit pull was three weeks stale, and his paid-off collections account had already cleared in that time."
Why this works
A stale record and a genuine risk factor look identical until someone rules out the boring explanation first.
Stage 5
Name the cause candidates
Say it like this
"Three named possibilities, not one vague flag: stale credit data, incomplete income documents, or a missing landlord reference. Tobias's was the first one."
Why this works
A person can push back on a named cause. They can't push back on "additional review needed."
Stage 6
Give the evidence test
Say it like this
"One button: 'Request a fresh credit pull.' If the flag was caused by stale data, this clears it in minutes. If it doesn't, that tells us the real cause is something else."
Why this works
TRACE's strongest move: one specific check that actually separates the top hypotheses, not a form to fill out and wait.
Stage 7
Say what you'd measure, and close
Say it like this
"I'd watch what share of flags trace back to stale data specifically, since that number tells you how much of your 'risk' flag rate is actually just an outdated file. An audit screen isn't a courtesy. It's the difference between a flag and an explanation."
Why this works
Closes on the direct answer, with a concrete number ready for a follow-up question.
Let's learn
Havenwell's screening assistant reviews a rental applicant's credit, income, and rental history, then flags roughly 18% of applications for a property manager's closer look.
Before the assistant existed, a property manager reviewed each application by hand, about twenty-five minutes, cross-checking credit, income, and references one by one. With the assistant, that drops to under a minute, and property managers report high satisfaction with the speed.
Knowledge spark: what's a stale credit pull?
A credit report that was fetched days or weeks before it's actually used to decide something. Balances change, accounts get paid off, and an old snapshot can describe a version of someone's finances that no longer exists.
Until now, a flagged applicant like Tobias got exactly one message: "Additional review needed." No factor, no reason, no next step.
Outcome for flagged applicants, opaque message versus audit view
The underlying applications didn't change. What changed was whether the applicant had anything to actually do with the flag.
The turn: the flag itself wasn't usually wrong. Most flagged applicants genuinely had something worth a closer look. The real problem is that "additional review needed" gave the applicant nothing to check, correct, or push back on, whether the flag was right or not.
A flag with no cause named isn't a decision an applicant can respond to. It's just a locked door with no sign on it.
At its worst: Tobias, whose finances were genuinely fine, a paid-off collections account that had already cleared, loses the apartment to another applicant while waiting on an appeal process nobody could tell him how to start. Havenwell's own data was already correct, just three weeks out of date, and nobody had built a way to say so.
The decision I would take back
We kept no persisted, timestamped record of exactly which inputs, and their ages, drove a specific flag. That made sense while flag volume was low enough that a support agent could just re-run the file by hand if a dispute came in. It stopped making sense once volume grew and "just re-run it" stopped being something anyone actually had time to do.
Share of flags traced to stale data, by week since the freshness check shipped
Nearly a third of all flags in week one traced back to nothing more than an out-of-date report. That share kept falling once refreshing stopped depending on a person remembering to do it by hand.
What I would leave alone: for an application that's approved outright, with no flag at all, the current one-line "approved" message is genuinely fine as-is. There's no decision to trace when nothing was ever in question.
The lesson: an audit feature isn't a courtesy screen bolted onto a decision. It's the actual difference between telling someone what happened and just telling them no.
Now here is the same thing as a story
The short version above is what you'd say defending this design to Havenwell's product leadership. Read this one for how the gap actually got found.
Tobias Reyes had good credit, steady income, and two years of on-time rent payments at his last place. He'd never worried about qualifying for an apartment before.
He applied to a unit through Havenwell on a Tuesday. By Wednesday, his status read "Additional review needed," with nothing else on the screen. No factor, no timeline, no next step.
Same underlying fact, two very different screens. Only one of them gave Tobias anything to do.
He wasn't the only one. Two friends from his same apartment search, applying to different units through Havenwell that same week, got the identical vague message. All three assumed, separately, that something specific in their own history had triggered it. None of them could say what.
That was the trigger, small and plain: two friends, then Tobias, all flagged the same way, all with nothing to go on. He called Havenwell support and asked, flatly, "what exactly triggered this?"
The whole story sits inside one gap: a report pulled on day one that was already three weeks old before Tobias ever applied.
The support agent who took his call did something that hadn't been standard practice: she manually re-ran his file end to end. The credit report behind his flag had been pulled three weeks earlier, before his collections account had finished clearing. A fresh pull would have shown a clean file.
Four very different reasons hide behind one flag. Tobias's case was the cheapest one to fix, and the one nobody had a button for.
Tobias didn't need a better explanation of the model. He needed to know his own file was simply three weeks behind his own life.
Here's the decision I'd take back. We never persisted which specific data, and its age, drove a given flag, because manually re-running a file was rare enough that a support agent doing it by hand felt like a reasonable stopgap. It stopped being reasonable once that manual re-run was the only thing standing between a real applicant and a lost apartment.
Four parts, and the old "additional review needed" message had none of them.
I'd build the audit view directly into the applicant's own status page: a timeline of what happened and when, a data-freshness check that runs automatically before any judgment is shown, the named cause candidates, and one button that does the actual fix if the cause turns out to be data, not risk.
The factors worth showing first are the ones that are both big and something an applicant can actually go check.
Replay the same Tuesday application under the new design: Tobias sees "Flagged: credit report is 21 days old" the moment his status updates, taps "Request a fresh pull," and clears within the hour, well before the unit goes to anyone else.
We built the flag message to be short because short felt clean and simple to maintain. It took three friends comparing identical, useless messages in one group chat to see that short and useless were the same thing here.
TRACE, run on one file instead of one metricNot a diagnosis of a falling number. TRACE is what turns "additional review needed" into something an applicant can actually act on.
T
Timeline. When each input actually mattered.
Application submitted day one, credit pulled that day from an already-stale report, flagged day two.
Puts the actual sequence in front of the applicant, not a single frozen verdict.
R
Recut. Slice the decision by factor.
Credit, income, and reference each shown as their own slice, so a thin file doesn't hide behind one average-looking score.
A decision made of several parts is only auditable when you can see each part separately.
A
Assume nothing about the data first.
The screen checks whether the credit pull is current before showing any judgment about risk. Tobias's was three weeks stale.
The hardest step, and the one that separates a real risk from a bookkeeping problem.
C
Cause candidates, named.
Stale credit data, incomplete income documents, or a missing landlord reference, not one vague flag.
A person can respond to a named cause. They can't respond to a locked door.
E
Evidence test. One button that actually checks.
"Request a fresh credit pull" either clears the flag in minutes or confirms the real cause is something else.
TRACE's strongest move: a specific test, not a form and a wait.
Checked in this order, because the cheapest, most common explanation should always get ruled out first.
The recap, one line per letter: timeline is the exact sequence from application to flag, recut is slicing the decision by factor instead of one score, assume nothing is checking data freshness before judgment, cause candidates is naming the real handful of reasons, and evidence test is the one button that actually resolves the ambiguity.
And if you want to be sure it really works, try it somewhere elseSame five letters, a small-business loan platform instead of a rental marketplace. A different building, and the stale input is a bank statement, not a credit pull.
A small-business lending platform uses an assistant to screen loan applications, flagging thin files for a human underwriter. Mapped onto TRACE: timeline is when each document, a bank statement, a tax filing, a business license, entered the file relative to when the model actually scored it. Recut is slicing the decision into cash flow, credit history, and time in business, instead of one composite score. Assume nothing is checking whether the bank statement on file is the most recent one available, since a business owner who just had a strong month may be scored against last quarter's numbers. Cause candidates are a stale bank statement, a missing tax filing, or a genuinely short business history. Evidence test is a single "resubmit latest statement" action that either clears the flag immediately or confirms the real issue is something else, like an actual gap in business history that no new document can fix.
A rental unit and a business loan are very different stakes. The same question, "is the data even current," turns out to matter first in both.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "rule out stale data first, name the real causes, give one action that would fix it," and stop.
Cost: there's no engineering time to build the full audit view this quarter. Start with the cheapest fix: a data-freshness check alone, even before the timeline and factor breakdown exist.
The model gets better, for real: if the screening model's accuracy improves, that's still not a reason to skip the freshness check. A more accurate model scoring a three-week-old report is still scoring the wrong file.
Where people run it wrong.
They treat "auditability" as a single confidence score shown to the user, instead of a real trace of inputs and timing.
They assume every flag reflects genuine risk, and never check whether the underlying data was even current.
They give an explanation with no action attached, which just tells someone why they lost, not what they could do about it.
How to use it live. When someone asks how you'd design for a user auditing a decision, don't start with "show them the reasoning." Start with the boring question first: is the data behind this decision even still true, and could you check that before showing anything else.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits "design for a user who wants to audit an AI decision"?
Tap to flip
ANSWER
TRACE: timeline, recut, assume nothing, cause candidates, evidence test. It fits because auditing one person's decision is the same shape of problem as diagnosing a metric drop, aimed at one file instead.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Tobias Reyes, a rental applicant with genuinely good finances who got flagged with no explanation at all.
3 · ASSUME NOTHING
What's the first thing this design checks, before showing any judgment?
Tap to flip
ANSWER
Whether the underlying data, like the credit pull, is actually current. Tobias's report was three weeks stale.
4 · THE CAUSE CANDIDATES
Name the three named reasons a Havenwell application might get flagged.
Tap to flip
ANSWER
Stale credit data, incomplete income documents, or a missing landlord reference.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Keeping no persisted, timestamped record of exactly which inputs and their ages drove a specific flag.
6 · THE NUMBER
Fill in the blank: with the audit view, ___% of flagged applicants successfully clear review, up from 22%.
Tap to flip
ANSWER
68%. The underlying applications didn't change; only whether the applicant had something to act on did.
7 · THE REPLAY
Same Tuesday application, redesigned status page. What changes?
Tap to flip
ANSWER
Tobias sees "credit report is 21 days old" immediately, taps to request a fresh pull, and clears within the hour, before the unit goes to anyone else.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what's the stale input there?
Tap to flip
ANSWER
A small-business lending platform. There, the stale input is an out-of-date bank statement instead of a credit pull.
Check yourself Score: 0 / 0
Multiple choice
1. What does this answer say should be checked first, before showing an applicant any judgment about risk?
A. Whether the applicant has used Havenwell before.
B. Whether the underlying data, like a credit pull, is actually current.
C. Whether the applicant's credit score is above 700.
D. Whether the property manager has reviewed similar applicants before.
Show hint
Look at the "A" step of TRACE.
Show answer
B. A stale record and a genuine risk factor look identical until someone checks the date behind the data.
True or false
2. True or false: Tobias's flag was caused by a genuine, current risk in his financial history.
True
False
Show hint
Look at the story section: what the support agent found on the manual re-run.
Show answer
False. His flag traced back to a three-week-old credit pull; his collections account had already cleared by the time he applied.
Fill in the blank
3. Fill in the blank: without the audit view, only ___% of flagged applicants successfully cleared review.
Show hint
Look at the grouped bar chart.
Show answer
22%. Most of the rest simply gave up waiting, or lost the unit before an answer ever came.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at "the decision I would take back."
Show answer
Model answer: Keeping no persisted record of which inputs and their ages drove a flag. It made sense while manual re-runs were rare enough for a support agent to handle by hand.
Short answer, where it wouldn't matter
5. Name a case where the current simple "approved" message is genuinely fine, with no audit trail needed.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: An application that's approved outright with no flag at all. There's nothing to trace when nothing was ever in question.
Short answer, apply it yourself
6. Pick a time you were rejected or flagged by an automated system. If you could trace it back to one specific cause, what do you think it would have turned out to be?
Show hint
Think of a loan, job application, or account-verification decision that gave you little or no explanation.
Show answer
Model answer: Many people guess it was an out-of-date piece of information, like an old address or an outdated document, rather than a genuine problem with their situation.
Before you close the answer
Why this works
Tests whether you can design a real, checkable trace instead of a vague "transparency" screen, and whether you'd rule out the boring, common explanation, stale data, before reaching for anything about the model's judgment.
Follow-up traps
"Won't showing applicants the exact cause candidates let people learn to game the system?" Response: the causes named here, like data freshness or a missing document, aren't things you can fake convincingly; naming them helps a legitimate applicant far more than it helps someone trying to game the flag.
"Isn't a single 'request a fresh pull' button too simple for a real audit feature?" Response: simple is the point. It's the one action that actually resolves the single most common cause candidate, and it doesn't need to solve every possible cause to be worth building first.
If pressed
Havenwell's real freshness threshold isn't a single fixed number of days for every data type; a credit report is treated as stale past 14 days, while a landlord reference letter is treated as valid for up to 90, since the two kinds of information change at very different rates.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
