Critique a design where the human sees only the final output and not the reasoning.
AUDITa verdict nobody, including the reviewer, can actually check
Corvenna Retail built an internal tool called Returns Guard that scores high-value returns as likely fraud or likely legitimate. Yusuf Al-Bashir reviews the flagged ones on a shared tablet at the returns desk, and the screen shows him exactly one thing: a verdict and a percentage.
The direct answer
The real flaw isn't that the score might be wrong. It's that nobody, not Yusuf, not an internal audit, not a customer disputing the call, has any way to check whether it's wrong, because the reasons behind it, the eval set it was tested on, and the model version that produced it are all missing. A number with no way to verify it isn't a decision aid, it's a claim asked to be taken on faith.
Do this, in order
Show the specific reasons behind the verdict, not just the confidence percentage.Why: without a stated reason, neither the reviewer nor anyone auditing the decision later has anything to check the verdict against.
Name the eval set the confidence score was actually validated against.Why: a score validated against the model's own past predictions, not real confirmed outcomes, is measuring agreement with itself, not accuracy.
Log the exact model version and date behind every single decision.Why: without a version pin, a denial from three months ago can't be reproduced or defended if it's ever formally disputed.
Add a visible appeal path that reaches an actual reason, not a repeat of the same score.Why: right now, a customer who disputes a denial and asks why gets the same unexplained number back, which settles nothing.
Run a regular blind test: have a reviewer re-derive a sample of verdicts independently before seeing the AI's call.Why: this is the only way to catch a systemic problem with the score itself, instead of only catching individual disputes as they happen.
Leave the fast, low-friction path in place for the lowest-value returns, where a wrong call costs very little.Why: not every decision needs the full reasoning trail. The stakes should decide how much proof a verdict has to carry.
How to answer this, stage by stage
Nobody is grading whether you can spot that a score alone looks thin. They're grading whether you can name exactly what's missing, and prove it with a real test, not just a feeling.
Stage 1
Scope the critique to one real artifact
Say it like this
"I'll critique a real one: a retailer's returns-fraud screen that shows a reviewer only a verdict and a confidence percentage, nothing else."
Why this works
Keeps the critique grounded in one concrete design, not a hypothetical.
Stage 2
Say your structure out loud
Say it like this
"I'll use AUDIT. Ask who built it and why. Uncover the real eval set behind the number. Demand the version pin. Isolate what's missing. Test it yourself."
Why this works
Signals a method for judging trustworthiness, not just a gut reaction to the screen.
Stage 3
Name who benefits from the current design
Say it like this
"The team that built this screen is measured on average decision time. A verdict-only screen is fast to build and fast to use, so there's no one in the room whose job it is to ask whether it's actually right."
Why this works
Names the incentive behind the design, not just the design itself.
Stage 4
Name the exact thing missing
Say it like this
"No reasoning trail. No named eval set. No model version pin. Just a number, which means nobody, including the reviewer, can actually check whether a specific verdict was right."
Why this works
This is the direct answer, stated as a specific, checkable list, not a vague sense of unease.
Stage 5
Prove the critique with a real test
Say it like this
"Yusuf re-derived forty recent denials himself, blind, before looking at the AI's call. He disagreed with six of them. That's the critique made concrete, not just argued."
Why this works
Shows the critique isn't speculation, it's testable and was tested.
Stage 6
Name the decision behind the flaw
Say it like this
"At launch, the team deliberately stripped the reasoning trail to keep each decision under twenty seconds. That made sense when volume was the only metric anyone watched. It stopped making sense once verdicts started getting disputed."
Why this works
Names a specific, reversible product decision behind the flaw, not just a description of the flaw.
Stage 7
Say where this critique wouldn't apply
Say it like this
"For the lowest-value returns, where being wrong costs almost nothing, a fast verdict-only screen is fine. The full reasoning trail should scale with what's actually at stake."
Why this works
Shows judgment instead of demanding maximum rigor everywhere regardless of cost.
Stage 8
Close on the one line
Say it like this
"A verdict without a reason isn't a decision aid, it's a claim asking to be trusted on faith, and the fix is showing the reasons the model already has, not inventing new ones."
Why this works
Restates the direct answer in one breath, ready for a follow-up.
Let's learn
Picture six hundred high-value return decisions a day, and one screen that shows the exact same two things every single time: a word, and a number.
Returns Guard reads a return request, the item, the reason given, the customer's history, and produces a verdict: likely legitimate or likely fraud, with a confidence percentage attached. Before the tool, a reviewer read the whole case file, the item photos, the customer's past orders, the stated reason, and made the call themselves, slower, but every decision came with a reason attached to it.
Same underlying model, same underlying case. One screen leaves a person with nothing to check. The other leaves them something to hold onto.
Now Yusuf sees "DENY, 91%" and has to decide whether to uphold it, in about twenty seconds, with nothing else on the screen to weigh.
Here's the turn: the real problem was never that the model might be wrong sometimes. Every model is wrong sometimes. The problem is that this design makes being wrong invisible, because there's no reason attached to check against, no eval set named to trust, and no version recorded to reproduce the decision later if anyone ever asks.
Denials overturned on formal appeal, verdict-only design vs. a reasoning-shown pilot at a second warehouse
Nearly a four-fold gap in overturn rate, from the same underlying model, just because a reason was attached to the number.
At its worst, an entire quarter's worth of legitimate customers get wrongly denied a refund with no attached reason, and the company has no way to say, months later, exactly why any specific case went the way it did.
Every branch of the appeal path dead-ends at the same missing thing: a reason.
The decision that mattered
At launch, the team deliberately stripped the reasoning trail from the screen to keep each decision under twenty seconds, trading explanation for speed. That made sense when the only metric anyone tracked was decisions per hour. It stopped making sense the moment a denial could be formally disputed by a customer or reviewed months later by an auditor.
What I would leave alone: low-value returns under a small dollar threshold, where the cost of a wrong call is genuinely tiny, can stay on the fast, verdict-only path. Not every decision needs the full reasoning trail attached.
The lesson: a number that can't be checked isn't evidence, it's a claim, and a screen full of claims eventually meets someone who asks for proof.
Now here is the same thing as a story
The short version above is what you'd say presenting this critique to Corvenna's head of trust and safety. Read this one for the audit that actually forced the question.
Yusuf Al-Bashir has reviewed high-value returns for three years, and he's known on his shift for catching a fraud pattern in a stack of otherwise ordinary-looking cases nobody else flagged.
Knowledge spark: what's a version pin?
A record of exactly which version of a model, on which date, produced a given score. Models get quietly updated over time. Without a version pin, a decision from three months ago can't be reproduced or defended, because the model that made it may not even exist in that form anymore.
Corvenna's quality team pulled forty recent high-value denials for a routine review and asked Yusuf to defend each one. He opened the first case, saw "DENY, 91%," and realized, sitting there, that he genuinely couldn't explain why. He'd upheld a number, not a reason.
A verdict is a conclusion. A case file is what lets someone else check the conclusion. The screen only ever had the first one.
So he ran his own test. Before opening the AI's verdict on any of the forty cases, he read the customer's history and the stated return reason cold and made his own call first. Then he compared. On six of the forty, his independent read disagreed with the AI's denial.
Eight months between the design shipping and anyone formally testing whether its verdicts held up.
He raised the six disagreements with the quality team, not as proof the model was bad, but as proof the screen gave him no way to have caught this himself, on any of his regular shifts, before the audit forced the question.
Yusuf wasn't wrong to trust the screen. The screen was never built to let anyone, including him, check whether trusting it was actually earned.
With a reasoning trail added, a version pin logged per decision, and a monthly blind test built into the workflow instead of a one-off audit, the same kind of disagreement now surfaces within weeks, not eight months, and each flagged case comes with an actual reason a customer service rep can read back to a disputing customer.
The old screen asked "do you trust this number." The new one asks "can you check this claim."
The team built the verdict-only screen first because it was the fastest thing to ship. It took an internal audit and six disagreements out of forty to see that fast and trustworthy were never the same claim.
AUDIT, in one screenNot a lecture on transparency. AUDIT is what tells you whether a claim can survive being checked.
A
Ask who paid for it.
The in-house ML team, measured on average decision time, a metric the verdict-only screen excels at regardless of accuracy.
Names the incentive shaping the design, not just the design itself.
U
Uncover the eval set.
The 91 percent confidence figure was calibrated against the model's own past predictions, not independently confirmed fraud outcomes like chargeback resolutions.
Shows the number measures agreement with itself, not real accuracy.
D
Demand the version pin.
No model version or scoring date is recorded per decision, so a three-month-old denial can't be reproduced if it's ever formally disputed.
Names a specific, checkable gap, not a vague call for "more transparency."
I
Isolate what's missing.
No reasoning trail, no failure-case gallery, no calibration curve, no similar-case comparison. Only the number.
The hardest step, and the direct answer to the question.
T
Test it yourself.
Yusuf's blind re-derivation of forty recent denials, disagreeing with six, turning the critique from an opinion into a checkable result.
Proves the flaw is real, not just plausible.
Four gaps, and every single one of them was a choice, not an accident, made at launch to keep the screen fast.
The recap, one line per letter: ask is the team's own speed incentive, uncover is a circular eval against the model's own past calls, demand is the missing version pin, isolate is the absent reasoning trail, and test is Yusuf's blind re-derivation catching six disagreements out of forty.
Five questions, in order. None of them require trusting the vendor's own summary of their own work.
And if you want to be sure it really works, try it somewhere elseSame five letters, a hospital triage report instead of a retail returns screen. A completely different setting, the same missing reasoning.
Solmere Health Diagnostics sells a triage-support tool that reads incoming emergency intake notes and labels each patient "urgent" or "routine." Dr. Wendeline Osei, an emergency physician, sees only that label on her screen, with no supporting detail.
Mapped onto AUDIT: ask is that Solmere is paid per hospital deployment and measured on adoption speed, not on how defensible a single label is under review; uncover is that the "urgent or routine" label was validated against the vendor's own retrospective chart review, not an independent, blinded clinical study; demand is that the tool's build number changes silently with vendor updates and no version is shown per label; isolate is that there's no listed symptom pattern behind the label, no confidence range, and no comparison to a similar past case; test is Dr. Osei running her own blinded read on a sample of recent "routine" labels and finding two that her own clinical judgment would have escalated.
Swap "denial" for "routine," and the exact same missing reasoning shows up in a setting where the stakes are even higher.
Patient charts requiring escalation within 24 hours, labeled "routine" by the tool, before and after showing symptom reasoning
A single monthly miss remained even after the fix, a reminder that showing reasoning reduces this risk, it doesn't erase it.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "a verdict with no reasoning, no named eval set, and no version pin can't be checked by anyone, so demand all three before trusting the number," and stop.
Cost: there's no engineering budget to rebuild the whole interface this quarter. Say so honestly, and start with just logging the version pin per decision, the cheapest single fix, since reasoning and eval-set work can follow.
The model gets better, for real: even a genuinely more accurate model doesn't fix this critique, because the problem was never the accuracy number, it was that nobody could check it, and a better hidden number is still a hidden number.
Where people run it wrong.
They treat a high confidence percentage as proof of quality, without ever asking what it was actually validated against.
They add an appeal button that just re-shows the same score, mistaking the appearance of recourse for an actual reason.
They wait for a lawsuit or a regulator to demand a version pin, instead of building one in from day one when it costs almost nothing.
How to use it live. When someone hands you a design that shows only a final output, ask one question first: if someone challenged this exact decision tomorrow, could anyone in the company actually reproduce and defend it? If the honest answer is no, that's the critique, stated plainly.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits an "critique this artifact" question?
Tap to flip
ANSWER
AUDIT: ask who paid, uncover the eval set, demand the version pin, isolate what's missing, test it yourself.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Yusuf Al-Bashir, a returns-fraud reviewer at Corvenna Retail, three years on the job.
3 · THE ARTIFACT
What does the critiqued screen actually show?
Tap to flip
ANSWER
A verdict, likely fraud or legitimate, and a confidence percentage. Nothing else: no reasons, no eval set, no version.
4 · ISOLATE
What's the exact thing missing from this design?
Tap to flip
ANSWER
A reasoning trail, a named eval set, a model version pin, and a failure-case gallery. All four, not just one.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Deliberately stripping the reasoning trail at launch to keep every decision under twenty seconds, when volume was the only metric that mattered.
6 · THE NUMBER
Fill in the blank: Yusuf's blind re-derivation of 40 denials disagreed with the AI's call on ___ of them.
Tap to flip
ANSWER
6, or 15 percent of the sample, all upheld without anyone able to check the reasoning behind them.
7 · TEST IT YOURSELF
How was this critique proven, not just argued?
Tap to flip
ANSWER
Yusuf read each case cold and made his own call before seeing the AI's verdict, then compared, turning a design critique into a checkable result.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what did the blind test find?
Tap to flip
ANSWER
Solmere Health Diagnostics' triage label. Dr. Wendeline Osei's blind read found two "routine" labels her own judgment would have escalated.
Check yourself Score: 0 / 0
True or false
1. True or false: the main problem with Returns Guard's screen is that its confidence score is probably too low.
True
False
Show hint
Look at the direct answer.
Show answer
False. The problem isn't the score's accuracy, it's that nobody can check whether it's accurate at all, because the reasons, eval set, and version are all missing.
Multiple choice
2. Why does it matter that the 91 percent confidence figure was validated against the model's own past predictions?
A. It means the number was calculated incorrectly.
B. It means the score measures agreement with the model's own history, not real, independently confirmed accuracy.
C. It means the model is definitely biased against certain customers.
D. It means the score updates too frequently to be useful.
Show hint
Look at the "uncover" step.
Show answer
B. A confidence number validated in a circle, against its own past calls, tells you the model agrees with itself. It doesn't tell you whether it's actually right.
Fill in the blank
3. Fill in the blank: under the verdict-only design, ___ percent of appealed denials were overturned, compared to 6 percent under the reasoning-shown pilot.
Show hint
Look at the bar chart in Section 1.
Show answer
22 percent. Nearly a four-fold gap, from the exact same underlying model, just because a reason was attached to the number.
Short answer, apply it yourself
4. Think of a score or verdict you've been shown by any app, a credit decision, a content moderation strike, a spam filter. What question would you have needed answered to actually check whether it was right?
Show hint
Think about what you wanted to know beyond the number itself when you disagreed with a call.
Show answer
Model answer: A common one: a content moderation strike with no cited rule or example, leaving no way to know which specific post or phrase triggered it.
Short answer, where it wouldn't matter
5. Name a kind of return decision at Corvenna where the verdict-only screen is genuinely fine as is.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: Low-value returns under a small dollar threshold, where the cost of a wrong call is small enough that the full reasoning trail isn't worth the added friction.
Short answer, the number question
6. If Yusuf's blind test had found 0 disagreements out of 40 instead of 6, would the critique of the missing reasoning trail still hold? Why or why not?
Show hint
Think about whether the critique depends on the model being wrong.
Show answer
Model answer: Yes. The critique is about the design's unfalsifiability, not the model's current accuracy. Even a model that's right every time still needs a way to prove it, since accuracy can change and nobody would notice without a check.
Before you close the answer
Why this works
Tests whether you can separate "is this model accurate" from "can this decision be checked," and critique the design for the second failure even without proof of the first.
Follow-up traps
"Isn't showing reasons just going to slow down decisions that were fine before?" Response: pair the fix with a stakes-based split, low-value returns keep the fast path, and reserve the full reasoning trail for decisions actually worth disputing.
"What if showing reasons just teaches fraudulent customers how to game the model?" Response: show the category of signal, like "address mismatch," not the exact underlying rule or weight, which keeps the reviewer informed without handing over a cheat sheet.
If pressed
Corvenna's actual pilot also stores a snapshot of the reasoning shown to the reviewer at decision time, separately from the live model, since the reasoning logic itself can be updated later and a stored snapshot is what actually makes a six-month-old decision defensible, not just the version number alone.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
