CaseAdvancedDesigning for Uncertainty & Trust / Designing for failure and graceful degradation / #17

Design the experience for a partial result rather than a failure.

SPARK design the batch to tell the truth about which files are actually fine

ClearRecord redacts and extracts records for Alder Bluff County Clerk's Office when someone files a public-records request. Here is the batch that beat Marcus Fennimore's afternoon, and the design that would have given nine of his twelve files back in the first twenty minutes.

The direct answer
Ship results per file, the instant each one finishes, sorted into three plain buckets: Done and ready to send, Needs you with the exact reason, and Still working. Give every done file's redactions one quick highlighted pass before it counts as ready. Never make nine finished files wait on the one that broke.
Do this, in order
  1. Ship results per file as they finish, never as one batch verdict.Why: nine finished files sitting behind one broken one is nine files held hostage for nothing.
  2. Sort into three plain buckets: done, needs you, still working.Why: a status that just says "processing" hides the exact decision Marcus needs to make right now.
  3. Name the specific reason a file needs a human, in plain words.Why: "review needed" tells him nothing. "Page 4, handwriting" tells him exactly where to look.
  4. Put every redaction a "done" file made in front of a human eye first.Why: a redaction tool that's confidently wrong is the worst realistic failure a public-records officer can ship.
  5. Hold off on a full diff viewer or automatic root-cause tagging on day one.Why: the three-bucket split and a plain reason already solve the real problem, the rest is polish.

How to answer this, stage by stage

Nobody is grading whether you know the words "partial result." They're grading whether you can say what happens to the files that were never actually broken.

Stage 1
Scope it to one batch
Say it like this
"I'll use ClearRecord, Alder Bluff County Clerk's public-records redaction tool, and a batch of twelve files where three couldn't finish cleanly."
Why this works
Gives you a real batch and a real number to design the screen around, not an abstract policy.
Stage 2
Say your structure out loud
Say it like this
"I'll use SPARK. Situation, payoff, anchor, risk, keep out."
Why this works
Signals you have a method before you start describing a screen.
Stage 3
Name today's situation, and the payoff you want
Say it like this
"Right now, any file the batch can't handle marks the whole thing failed, and Marcus has to open all twelve by hand just to find the one that broke. I want him to trust that whatever's actually done is already usable, so he stops re-checking files that were never broken in the first place."
Why this works
Separates a real design question from a vague "handle errors better" request.
Stage 4
Give the anchor, the one decision
Say it like this
"Results ship per file into three buckets: done and ready to send, needs you with the exact reason, and still working. Every done file's redactions are also highlighted for one quick pass before it's marked ready."
Why this works
This is the actual answer to the question, concrete enough to argue with.
Stage 5
Prove it survives being wrong
Say it like this
"Say the tool is confidently wrong about a redaction on a page it scored high confidence on. That's exactly why the highlight-and-check step sits on every done file, not just the ones it flagged as uncertain."
Why this works
Shows the design was built against its own worst realistic failure, not just described in the abstract.
Stage 6
Close on the one line
Say it like this
"Ship per file, in three plain buckets, with a human eye on every redaction before it's called done. Never make nine finished files wait on the one that broke."
Why this works
Restates the decision in one breath, the way you'd want to leave the room.

Let's learn

ClearRecord is an AI tool Alder Bluff County Clerk's Office uses to pull requested records out of a stack of scanned files and redact private information before a public-records response goes out.

Before ClearRecord, a records officer read and redacted every page by hand, about twenty-five minutes a file, with a multi-file request often taking days to turn around.

Now ClearRecord processes a whole batch automatically, most files finishing in under a minute each.

Hand sketched flow diagram titled Today, without this design. Four boxes: Batch submitted, One spinner runs, Any failure all fail, Clerk reruns all, with the last box emphasized.
Four steps, and the third one quietly erases whatever the first two got right.

Here's the turn: the extra speed was never the problem. The real cost showed up the moment a batch of twelve files had even one file the tool couldn't finish, a bad scan, handwriting, a foreign-language page, because the whole batch got marked failed, hiding the nine files that were already done and fine.

One batch of 12 files, by outcome
12 6 0 9 done 2 needs you 1 still working
Under the old design, all twelve of these sat behind one banner that just said "failed."

At its worst, a rushed team ships a "done" file with a redaction the model missed, and something that should have stayed private goes out with the response.

The decision I would take back ClearRecord's batch job was built to report status once, at the very end, for the whole batch, with no detail on which specific file failed or why. That made sense when most test batches finished cleanly end to end. It stopped making sense the moment real requests, with real scan quality problems, started failing partway more often than the test files ever had.

What I would leave alone: a single-file, one-page request doesn't need this treatment. There's no batch to split, so the current all-at-once handling is already fine there.

The lesson: a status that only ever says "done" or "failed" is a coin flip wearing the costume of information. The real question was always which nine, and which three, and why.

Now here is the same thing as a story

The short version above is what you'd say defending this redesign to Alder Bluff's county records board. Read this one for how the trust actually built up.

Marcus Fennimore has worked as a records officer for three years, and he knows exactly which fields state law requires redacted on a given record type without ever needing to look it up.

ClearRecord launched and the first months were good. Most requests turned around the same day instead of the same week, and Marcus's backlog dropped fast.

Knowledge spark: why would one bad file fail an entire batch job? Many batch tools are built to report success only when every item in the batch finishes cleanly. It's simpler to build one pass or fail flag for the whole job than to track and report the state of each item separately. That simplicity is exactly what turns one broken file into a false failure for every file next to it.

Month one, Marcus reviewed every file's redactions personally, regardless of what the batch status said. By month three he only reviewed files ClearRecord flagged as uncertain. By month five he stopped opening "done" batches at all, trusting the "Batch complete" banner on its face.

Hand sketched labeled parts diagram titled The results screen close up. A document icon at the center labeled Results screen, with four callouts around it: done ready, needs you, still working, redaction nudge.
This is what the redesigned screen shows Marcus. The old one only ever showed one word.

A batch of fourteen files came in for a routine zoning-dispute request. One file, a scan creased hard down the middle, came back unreadable. The screen said only "Batch failed." Marcus, reasonably, from five months of experience with that message, assumed all fourteen needed his attention.

Thirteen files were already redacted correctly and fully done. The screen just never said so.

He opened all fourteen files by hand that afternoon, confirming that thirteen of them had, in fact, been right the whole time, and losing most of the afternoon to work that never needed doing.

Hand sketched comparison titled The day it is wrong. Left, a red box icon labeled Missed redaction, caption confidently wrong. Right, a teal person icon labeled Sample nudge catches it, caption a human eye passes.
The failure this design actually has to survive isn't a broken file. It's a done file that's confidently wrong.

It wasn't really about the lost afternoon. It was that "failed" meant nothing specific, so every batch failure from then on cost Marcus a full manual re-check of every file in it, exactly the habit ClearRecord was supposed to end.

Files usable so far, old design versus new, during one 12-file batch
12 6 0 failed, min 47, 0 usable min 20, 9 usable Minute 0 Minute 47
The old line sits flat at zero for the entire batch, then arrives at a failure. The new one gives Marcus nine usable files before the old design had said anything at all.

With the redesigned screen, the same fourteen-file batch shows thirteen files landing in "Done, ready to send" the moment each finishes. One file lands in "Needs you: page torn, text unreadable" within the first four minutes. Marcus reviews just that one file, confirms the highlighted redaction sample-check on the thirteen in under ten minutes total, and sends the full response by early afternoon instead of losing it.

The old design asked Marcus to trust a single word for the whole batch. The new one tells him exactly which files earned that trust and which one didn't.

I built the batch job to report status once, at the end, because that's the simplest thing to build and test batches almost always passed. It took one creased scan, and an entire wasted afternoon confirming work that was already right, to see that "simple to build" and "tells the truth" were never the same thing.

The anchor, and what protects itNot a lecture on staying calm about failures. SPARK is what tells you which decision the whole screen actually hangs on.

S
Situation. How this happens today, without the redesign.
Any file the batch can't handle marks the whole thing failed, and Marcus opens every file by hand to find which one broke.
Grounds the whole design in a real, wasted afternoon, not a hypothetical one.
P
Payoff. The habit this should build.
Marcus trusts that whatever's marked done is already usable, and stops re-checking files that were never broken.
Names the real behavior change the design is trying to produce.
A
Anchor. The one decision everything hangs on.
Results ship per file into three buckets, done, needs you, still working, with every done file's redactions highlighted for one quick pass first.
This is the hardest step and the answer to the question: a concrete, arguable design decision.
R
Risk. What breaks the first time it's wrong.
A done file with a confidently missed redaction. The highlight-and-check step has to sit on every done file, not just the flagged ones, since this is a public release, not a private draft.
Proves the anchor was designed against its own worst realistic failure, not just described.
K
Keep out. What we won't build, day one.
No full diff viewer, no automatic root-cause tagging for every failure type. The three-bucket split and a plain reason already solve the real problem.
Shows judgment about what stays out, not just a wish list of what's in.

The recap, one line per letter: situation is a batch that fails as one unit and a wasted afternoon, payoff is teaching Marcus to trust "done" without re-checking it, anchor is the three-bucket screen with a mandatory redaction pass, risk is a confidently wrong redaction on a file marked done, and keep out is holding back a diff viewer and auto root-cause tagging on day one.

Hand sketched quadrant titled Which files need a human look. Axes AI confidence and risk if wrong. Clean scan sits high confidence low risk. Handwriting page sits low confidence medium risk. SSN near fold sits medium confidence high risk. Foreign language sits low confidence medium risk.
The redaction pass isn't just for the files ClearRecord admits it's unsure about. The top-right corner is exactly where a confident miss does the most damage.
Hand sketched icon list titled What we left for day two. Three items: a funnel icon labeled auto root cause tagging, a document icon labeled a full diff viewer, a box icon labeled cross file comparison.
Each of these is a real feature someone will ask for eventually. None of them belongs in the first version.
Hand sketched timeline titled One batch start to finish. Four milestones: Batch starts 0 minutes, 9 files done 20 minutes highlighted, 3 files flagged 20 minutes, Clerk reviews same afternoon.
Twenty minutes after starting, Marcus already has nine files he can act on.

And if you want to be sure it really works, try it somewhere elseSame five letters, a hospital discharge-translation batch instead of a records office. A different risk, the same three buckets.

Kestrel Valley Medical Center uses Transcura, an AI tool that translates multi-page discharge instructions for patients who don't read English as a first language. Wren Oduya, a discharge coordinator, runs it on a patient's twenty-two page instruction packet before they leave the hospital. Mapped onto SPARK: situation is a coordinator today, printing instructions in English and waiting on a staff interpreter's availability, sometimes hours, before a patient can safely leave; payoff is the habit to build, trusting that translated pages already marked done are safe to hand over immediately, instead of holding the entire packet until every page clears.

The anchor here is structurally the same idea, aimed at a sharper risk: pages ship per file into the same three buckets, with medication dosage and timing instructions specifically highlighted for a human bilingual staff member's pass before any page counts as done, regardless of how confident Transcura's translation model reads. The risk Kestrel Valley's team designed against was a confidently wrong dosage translation on a page marked done, since a missed decimal point in a medication instruction is a materially different kind of harm than a missed comma in a public-records redaction, and needed its own dedicated check rather than the general sample nudge alone.

Hand sketched decision tree titled A hospital translation batch's triage rule. Root: page fails auto translate. Four branches: handwriting found leads to flag for human, unsupported language leads to flag for human, low confidence only leads to retry once, high confidence leads to ship as done.
Same shape of rule as ClearRecord's buckets, tuned for a different kind of page that can't afford to be wrong.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "ship per file into done, needs you, and still working, with a mandatory check on every done file before it counts as safe," and stop.
Cost: there's no time to build the full three-bucket UI this sprint. Say so honestly, and ship the plain per-file reason text first, since that alone stops the worst of the wasted re-checking.
The model gets better, for real: if ClearRecord's redaction accuracy genuinely climbs to near perfect, that's still not a reason to drop the mandatory human pass on done files, the one confident miss is exactly as costly as it ever was, however rare it becomes.

Where people run it wrong.
They build a progress bar for the whole batch and call partial visibility solved, without ever splitting out which specific files are actually usable.
They only apply a human check to files the model flagged as uncertain, missing the confidently wrong ones entirely.
They assume "processing" is informative, when it hides the one decision the person actually needs to make right now.

How to use it live. When someone asks you to design a partial-result experience instead of a failure, ask yourself one question before sketching a single screen: if this batch had ten good files and one bad one right now, would the person in front of it be able to tell which is which? If the honest answer is no, you haven't designed a partial result, you've just relabeled a failure screen.

Flashcards (tap any card to flip it)

1 · THE FRAMEWORK
What framework fits a "design the partial-result experience" question?
Tap to flip
ANSWER
SPARK: situation, payoff, anchor, risk, keep out. Ground the anchor in today's all-or-nothing pain, then prove it survives a confident miss.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Marcus Fennimore, a records officer at Alder Bluff County Clerk's Office, who knows the required redactions for a record type without looking them up.
3 · THE SITUATION
How did a broken file get handled today, without this redesign?
Tap to flip
ANSWER
One bad file marked the entire batch "failed," with no detail on which file broke or why, so Marcus had to open every file by hand to find out.
4 · THE ANCHOR
What's the one design decision this answer hangs on?
Tap to flip
ANSWER
Ship results per file into three buckets, done, needs you, still working, with every done file's redactions highlighted for a human pass first.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Reporting batch status once, at the end, for the whole job, with no detail on which specific file failed or why.
6 · THE NUMBER
Fill in the blank: in the redesigned batch, ___ of 12 files landed in "Done, ready to send."
Tap to flip
ANSWER
9 files. Only 2 needed Marcus's attention, and 1 was still working, none of which the old all-or-nothing status could ever have shown him.
7 · THE REPLAY
Same fourteen-file batch, redesigned screen. What changes?
Tap to flip
ANSWER
Thirteen files land in "Done" as they finish, one lands in "Needs you" within four minutes. Marcus reviews just that one file and sends the response by early afternoon.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what's the anchor there?
Tap to flip
ANSWER
Transcura, Kestrel Valley Medical Center's discharge-translation tool. The anchor is the same three buckets, with medication dosage lines getting their own mandatory human check, since a wrong dosage is a sharper harm than a missed redaction.

Check yourself Score: 0 / 0

Short answer, apply it yourself
1. Think of a tool you've used that processes several things at once, an upload, an import, a batch job. When one part failed, did you find out which part, or just that "something went wrong"?
Show hint
Ask whether it named a specific item, or just reported one status for everything.
Show answer
Model answer: Most batch tools report one overall status, hiding exactly which item actually failed, the same gap this answer redesigns around.
Multiple choice
2. Why did Marcus lose most of an afternoon on a batch where only one of fourteen files was actually broken?
  • A. ClearRecord deleted the thirteen good files by mistake.
  • B. The batch reported one overall "failed" status, with no detail on which file broke.
  • C. Marcus was unfamiliar with the redaction requirements.
  • D. The request was too large for ClearRecord to process at all.
Show hint
Look at the key point block, "The decision I would take back."
Show answer
B. A single all-or-nothing status hid which one file actually needed attention, so Marcus had to check all fourteen by hand.
True or false
3. True or false: under the redesigned screen, files that are marked "done" skip any human check entirely.
  • True
  • False
Show hint
Look at the R step, risk, and the anchor description.
Show answer
False. Every done file's redactions get a quick highlighted human pass before it counts as ready, since a confident miss is the worst realistic failure here.
Fill in the blank
4. Fill in the blank: in the redesigned batch of 12 files, ___ files landed in the "Done, ready to send" bucket.
Show hint
Look at the stacked bar chart.
Show answer
9 files. With 2 needing a human look and 1 still working, none of which the old status could show separately.
Short answer, where it wouldn't matter
5. Name a place in ClearRecord where this same three-bucket redesign genuinely doesn't need to apply.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: A single-file, one-page request. There's no batch to split apart, so the existing all-at-once handling is already fine there.
Short answer, name the reversal
6. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at the key point block titled "The decision I would take back."
Show answer
Model answer: Reporting batch status once, at the end, with no per-file detail. It made sense when test batches almost always finished cleanly, so a single pass or fail flag seemed like enough.
Before you close the answer
Why this works
Tests whether you'll design around what's actually usable right now, or just relabel a failure screen with softer words. Most candidates stop at "show a progress bar" without ever splitting out which specific files are done.
Follow-up traps
"Doesn't a mandatory check on every done file slow the whole batch down?" Response: it adds a light, quick pass, not a full re-review, and it's worth it since a confident miss is a far more serious failure than a few extra seconds per file.

"What if 'needs you' becomes a dumping ground nobody actually reviews?" Response: give it its own visible count and plain reason per file, the same discipline that stops "processing" from becoming a meaningless catch-all.
If pressed
ClearRecord's redaction sample-check specifically re-renders each flagged region at full resolution before showing it to Marcus, since a redaction that looks complete at thumbnail size has, in testing, sometimes left a corner of readable text at full zoom.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more