ConceptAdvancedDesigning for Uncertainty & Trust / Trust, transparency and explainability in UX / #9
What transparency is legally required versus merely good practice?
GUARD the product is Cobalt Screen, an AI tool that ranks and auto-rejects hourly job applications for Bracknall Retail
Bracknall Retail is a regional chain, about 180 stores. Coretta Nyambura runs recruiting operations for hourly staff and has used Cobalt Screen, a vendor tool that ranks and auto-rejects applications, since it went live two years ago.
The direct answer
Tell every applicant, before they apply, that an automated tool is deciding whether their application moves forward. If it screens someone out, log and give them the one specific reason category behind that no, on request. Publish an independent bias audit of the tool's selection rates once a year. Explaining the full scoring model, or giving every rejected applicant a live human reviewer, is good practice worth doing. It is not the law.
Do this, in order
Disclose that an automated tool is deciding, before anyone applies.Why: an applicant can't ask for a human review of a decision they don't know a machine made.
Log a specific reason category for every automatic rejection.Why: this is the exact piece Bracknall never had on file, and the piece a regulator can actually demand.
Publish an independent bias audit of selection rates by group, on a fixed yearly schedule.Why: several hiring-tech laws already require this specific thing, not a one-time launch check.
Offer a path to request human review, with a real deadline attached.Why: a right to contest only matters if someone knows it exists and gets an answer back.
Save the full score breakdown and reapply tips for later, as good practice, not a requirement.Why: worth building eventually, but building it first would have delayed the two things actually owed by law.
Never publish the model's internal weights or ranking logic.Why: no law asks for that, and it only teaches people to game a résumé instead of doing the job.
How to answer this, stage by stage
Nobody is grading whether you know every regulation by name. They're grading whether you can split "must" from "should" and defend the line.
Stage 1
Scope it to one real system
Say it like this
"I'll answer this for Cobalt Screen, an AI tool that ranks and auto-rejects hourly job applications for a retail chain."
Why this works
Turns a legal question that could span every AI product into one you can actually reason about.
Stage 2
Say your structure out loud
Say it like this
"I'll use GUARD. Groups affected, where the harm lands unequally, who can't contest it, the specific reduction, and how you'd detect it."
Why this works
Signals you're separating a legal-risk question from a general "be more transparent" essay.
Stage 3
Reframe the question
Say it like this
"This isn't really 'how transparent should we be.' It's 'which piece of transparency does a rejected applicant, or a regulator, have an actual right to demand.'"
Why this works
Separates the answer from a values essay and grounds it in who can actually make a claim.
Stage 4
Give the one decision
Say it like this
"Disclose the AI, log a reason for every reject, publish a yearly bias audit, and offer human review on request. Everything past that, like a full score breakdown, is a nice add, not a requirement."
Why this works
Matches the direct answer word for word, so there's no gap between what you say first and what you defend later.
Stage 5
Prove it with a failure
Say it like this
"Bracknall auto-rejected thousands of applicants with an identical form email and nothing on file about why. When a state inquiry asked about one of them, nobody, not even Bracknall, could answer. That's not a model problem. That's a missing record."
Why this works
Shows the legal risk lives in what wasn't recorded, not in whether the model made a mistake.
Stage 6
Say what you'd measure
Say it like this
"I'd watch the selection-rate ratio by protected group every quarter, not just once a year at audit time, since that's the number that actually predicts a legal problem before it becomes one."
Why this works
Shows you think about ongoing risk, not a one-time compliance checkbox.
Stage 7
Close on the one line
Say it like this
"Legal transparency is about giving the powerless person a paper trail and a lever. Good practice is everything past that. Don't confuse generosity with obligation, and don't skip the obligation while you build the generosity."
Why this works
Restates the core split in one breath, ready for whatever follow-up lands next.
Let's learn
Here is what the applicant on the other end of an automatic no is actually owed, and here is everything past that which is simply a kindness.
Cobalt Screen reads every hourly-job application Bracknall Retail gets and ranks or auto-rejects it before a person ever opens the file.
Knowledge spark: what's a bias audit, in this sense?
A test comparing how often the tool advances applicants from different groups. If one group clears the bar much less often than another, with no job-related reason, that's the signal a regulator looks for.
Before Cobalt Screen, Bracknall's recruiters hand-screened about 3,400 hourly applications a month across 180 stores, roughly six hours a week per store manager. Now the tool auto-rejects about 72% of applicants in under a second, and recruiters only look at the remaining 28%.
Applicants with a specific reason on file for their rejection
Not one of the roughly 2,450 monthly auto-rejects had a reason on file before the fix. Every one does now.
The turn: the extra mistakes an automated tool makes are not really the risk here. The real risk is that Bracknall kept no record at all of why any of those 2,450 people a month were screened out, so nobody, not even Bracknall, could ever say what happened to one of them.
The decision I would take back
We sent every auto-rejected applicant the identical templated line, "we've moved forward with other candidates," with nothing behind it. It felt simple and safe: say as little as possible, so there's nothing to argue with. It made sense while nobody was asking. It stopped making sense the day a regulator did.
What I would leave alone: the model's internal scoring weights and ranking logic. No law asks Bracknall to publish those, and doing it would only teach applicants to game the résumé instead of doing the job well.
We didn't need a better model. We needed a reason on file for every single "no."
The lesson: legal transparency almost never asks you to explain your system perfectly. It asks you to leave the person on the receiving end a paper trail and a lever. Everything past that is a choice, not an obligation, and building the choice first while skipping the obligation is exactly backwards.
Four things, and Bracknall had built none of them when the inquiry landed.
Now here is the same thing as a story
The short version above is what you'd say defending this split to a room of lawyers. Read this one for how it actually got found.
The night desk isn't where this story happens. It happens at 9am on a Thursday, when Coretta opens her dashboard and, for six months running, watches Cobalt Screen quietly do a job she used to do by hand across 180 stores.
She could once eyeball a stack of forty paper applications in an hour and know, almost without reading the second page, who to call back. She trusted her own read on people. Cobalt Screen didn't replace that instinct. It just moved it upstream, ranking the flood before she ever touched it.
Coretta has a dashboard. The applicant on the other end has a form letter and nothing else.
For a long while, that was fine. Coretta reviewed the 28% who cleared the bar, filled the roles, moved on. She stopped even glancing at the rejected pile, since the tool had never once flagged something that later turned out to matter.
Then a state civil rights office opened an inquiry, after a complaint from an applicant group at three stores, and asked Bracknall a plain question: why was this specific applicant screened out, and what does your data show across applicants like them.
Coretta went looking for the file. There wasn't one. Cobalt Screen had made a decision on every one of those 2,450 monthly rejects and kept no record beyond a single pass/fail flag with no reason attached.
The inquiry didn't ask whether the tool was biased. It asked for a file that simply didn't exist.
What that cost, at its worst: the whole hiring pipeline paused company-wide for four weeks while legal and engineering tried to reconstruct, applicant by applicant, what the tool had likely done, with no real way to prove any of it either way.
I built that templated rejection because it looked clean and defensible in a planning review. Saying less felt like saying nothing wrong. It took a regulator's plain question, one we genuinely could not answer, to see that silence isn't neutral. It's just a decision with no paper trail behind it.
Bracknall had built the left branch only. The right one is what the law actually asks for.
With reason logging and disclosure shipped, the same kind of inquiry now closes in about 9 days instead of dragging for a month: Coretta pulls the applicant's reason code, the audit numbers for that quarter, and sends both over before lunch.
The old process asked nothing of Bracknall except silence. The new one asks Bracknall to write down, every single time, why it said no.
GUARD, the two people this legal question is really aboutNot a fairness lecture. GUARD is what forces you to name who can push back and who genuinely can't.
G
Groups. Who's affected.
Coretta, who holds the dashboard, and every applicant Cobalt Screen auto-rejects with no explanation.
Names the operator and the subject by name, not "stakeholders."
U
Unequal. Where the harm lands.
Applicants with no relationship to Bracknall beyond one submitted form, no insider to ask, no way to know a machine decided.
Points at exactly who has the least power to notice or push back.
A
Ability to contest. The hard step.
An applicant who never learns AI screened them can't request a human review of a decision they don't know exists.
The design decision that quietly took the lever away in the first place.
R
Reduce. The specific fix.
Disclose the tool, log a reason code per reject, publish a yearly bias audit, offer human review on request.
A product decision, not a policy memo nobody reads.
D
Detect. How you'd know.
Quarterly selection-rate ratio by group, watched before the annual audit, not instead of it.
Catches a drifting number months before a regulator has to ask about it.
Quarterly selection-rate ratio, one applicant group, five quarters
Quarterly checks caught the Q3 dip below 0.80 three months before the annual audit would have. That's the detect step doing real work.
The recap, one line per letter: groups is Coretta and every applicant Cobalt Screen touches, unequal is the applicant with no insider and no way to know, ability to contest is the lever that was never handed over, reduce is the four-part disclosure fix, and detect is the quarterly ratio that catches a dip early.
And if you want to be sure it really works, try it somewhere elseSame five letters, an online marketplace's refund tool instead of a hiring tool. A different industry, and this time the subject is a customer, not an applicant.
Wrenfield Partners runs an online marketplace. Ledgerbrook is the AI tool that auto-approves or auto-denies refund requests under a set dollar threshold. Halima Osei leads the disputes team.
Mapped onto GUARD: groups is Halima's team, who can escalate any case they want, and the customer whose refund got auto-denied with one line, "does not meet our refund policy." Unequal is that a customer disputing a 40 dollar order has no realistic way to escalate, while a customer disputing a 4,000 dollar order calls in and gets a human immediately. Ability to contest is the gap: small-dollar denials had no visible path to a person at all. Reduce is a specific fix: every denial gets a one-line policy-clause reference, plus a single tap to request human review regardless of order size. Detect is a monthly check on the auto-denial rate by order size band, to catch the tool quietly tightening against cheaper orders over time.
The cheap, high-risk-if-skipped moves are exactly the ones worth building first, in either company.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "disclose it, log a reason, audit yearly, offer review on request, nothing else is required," and stop.
Cost: there's no budget this quarter for a full audit program. Say so honestly, and ship the reason logging and disclosure first, since those cost far less and carry more of the actual legal exposure.
The model gets better, for real: if Cobalt Screen's accuracy improves next quarter, that changes nothing about what's legally owed. A more accurate model can still auto-reject someone with zero record of why.
None of these four are owed by law anywhere Bracknall operates. All four are still worth doing, once the minimum is shipped.
Where people run it wrong.
They treat "we might get sued" as the whole analysis, instead of naming the specific disclosure, record, and audit a law actually names.
They build the generous version first, a full explainability dashboard, and never ship the boring required version, a reason code on file.
They assume good practice and legal requirement are the same size of project, when the legally required piece is almost always the cheaper one.
How to use it live. When someone asks what transparency is "required," ask yourself: could this specific applicant, or a regulator standing in for them, point at a law and demand this exact thing? If yes, it's required. If the honest answer is "it would just be nice," it's good practice, build it second.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits "what transparency is legally required vs. good practice"?
Tap to flip
ANSWER
GUARD: groups, unequal harm, ability to contest, reduce, detect. The ability-to-contest step is what separates a legal minimum from a nice-to-have.
2 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Coretta Nyambura, who runs recruiting operations for hourly staff across Bracknall Retail's 180 stores.
3 · THE HABIT
What did Coretta's team stop doing because the tool worked?
Tap to flip
ANSWER
They stopped even glancing at the rejected pile, since the tool had never once flagged something that turned out to matter.
4 · ABILITY TO CONTEST
What's the gap this answer turns on?
Tap to flip
ANSWER
Applicants who never learn AI screened them can't request review of a decision they don't know exists. No disclosure means no lever.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Sending every auto-rejected applicant an identical form email with no reason on file, to look simple and safe.
6 · THE NUMBER
Fill in the blank: about ___ applicants a month were auto-rejected with zero reason on file, before the fix.
Tap to flip
ANSWER
2,450 (72% of 3,400 monthly applicants). After the fix, 100% of those rejections carry a specific reason code.
7 · THE REPLAY
Same state inquiry, redesigned process. What changes?
Tap to flip
ANSWER
It closes in about 9 days instead of a month-long company-wide pause, because Coretta can pull a reason code and a quarter's audit numbers on request.
8 · CROSS PRODUCT TRANSFER
Section 4 runs GUARD again on a different product. Which one, and what's different about who can't contest?
Tap to flip
ANSWER
Wrenfield Partners' refund tool, Ledgerbrook. There, it's small-dollar customers who had no visible path to a human, while large-dollar disputes always got one.
Check yourself Score: 0 / 0
Multiple choice
1. Why does this answer name disclosure, reason logging, a bias audit, and human review as legally required, rather than a full score-breakdown dashboard?
A. Because a full dashboard is technically impossible to build.
B. Because those four are the specific things an applicant or regulator can point at a law and demand, while the dashboard is a generosity past that.
C. Because dashboards only matter for salaried roles, not hourly ones.
D. Because Cobalt Screen doesn't support building a dashboard.
Show hint
Look at the direct answer and the priority list's last two items.
Show answer
B. The legal minimum is what's demandable. Everything past it is a choice, even a good one.
True or false
2. True or false: this answer recommends publishing Cobalt Screen's internal ranking weights to applicants.
True
False
Show hint
Look at "what I would leave alone."
Show answer
False. No law asks for that, and publishing it would just teach applicants to game the résumé instead of doing the job.
Fill in the blank
3. Fill in the blank: the quarterly selection-rate ratio dipped to ___ in Q3, below the 0.80 legal threshold, before recovering.
Show hint
Look at the line chart of the selection-rate ratio.
Show answer
0.79. Checking quarterly instead of only once a year caught this three months before an annual audit would have.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at "the decision I would take back."
Show answer
Model answer: Sending an identical, reason-free rejection email to every auto-screened applicant. It made sense while nobody was asking why, and it stopped making sense the day a regulator did.
Short answer, where it wouldn't matter
5. Name something in this answer that is genuinely optional, not legally required, even though it would help applicants.
Show hint
Look at the labeled-parts diagram, "What is good practice, not the law."
Show answer
Model answer: A full score breakdown, reapply tips, live chat support, or auditing in every state before any law requires it there. All good, none required.
Short answer, apply it yourself
6. Pick an AI product you've used that made a decision about you (a loan, a job application, a content flag). What's one piece of transparency you'd guess is legally required there, versus one that would just be a nice extra?
Show hint
Think about whether you could point at a specific law and demand the thing, versus just wishing the company offered it.
Show answer
Model answer: Most people land on "tell me AI was involved" and "give me a specific reason" as the required floor, and "explain the whole model to me" as the nice extra nobody's entitled to.
Before you close the answer
Why this works
Tests whether you can draw a hard line between a legal floor and a values-driven ceiling, instead of treating "transparency" as one soft, undifferentiated good.
Follow-up traps
"What if there's no specific law in your jurisdiction yet?" Response: build disclosure and reason logging anyway, since the record costs little and every serious jurisdiction is heading that direction; wait on the yearly published audit until it's actually required, and prioritize the cheap pieces first.
"Isn't publishing a bias audit itself a legal risk, since it might show a problem?" Response: no, the audit is the thing regulators already expect to see; not having one, or having one you hide, is the far bigger exposure.
If pressed
Bracknall's actual reason-code taxonomy has only six categories (minimum experience, required certification, availability mismatch, incomplete application, duplicate application, location mismatch), deliberately coarse so it's fast to log and hard to reverse-engineer into the model's exact scoring logic.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
