Design the interface for a user supervising three agents at once.

Corrowmoor County Department of Workforce Services runs three AI agents on every unemployment claim: one drafts eligibility from the application, one scores fraud risk, one checks submitted documents. Rowan Fitzgerald is a caseworker who has to supervise all three for every claim that crosses his desk.

How to answer this, stage by stage

Nobody is grading whether you can draw three panels on a screen. They're grading whether you know what a person can and can't hold in their head across three separate places.

Let's learn

Here's what happens when three helpful things all compete for the same five minutes of someone's attention.

Corrowmoor County's benefits office runs three separate AI agents on every unemployment claim. One reads the application and drafts an eligibility category. One scores the claim for fraud risk. One checks the uploaded pay stubs and ID against what's required. Rowan Fitzgerald reviews all three before a claim gets approved.

Same claim, three doors. Nothing in any one door shows what's behind the other two.

Before the agents, Rowan handled every part of a claim himself, reading the application, checking documents, watching for fraud patterns, one claim fully at a time. Slower, but never split across three separate places.

Now each agent has its own queue and its own screen. Rowan clears the eligibility queue in the morning, the document queue after lunch, the fraud queue whenever it fills up, each one entirely on its own.

Here's the turn: the three agents are each doing their job well. The problem shows up in the gap between them, the claim where the fraud agent raises a flag in one tab while the document agent quietly approves the same claim in a completely different tab, and nothing on either screen tells anyone to look at both.

What I would leave alone: a caseworker who genuinely needs to dig into one agent's full reasoning, say, exactly why the fraud model flagged a claim, still needs a detailed single-agent view. That view isn't the problem. Making it the only view is.

The lesson: three agents working well on their own can still fail together, in the gap between their screens, and no single agent's own dashboard was ever going to show that gap.

Now here is the same thing as a story

The short version above is what you'd say pitching this redesign to Corrowmoor's IT director. Read this one for how the gap actually cost someone.

Rowan Fitzgerald has worked benefits claims at Corrowmoor for six years. He's good at catching an inconsistent story in an application before he's finished reading it.

When the three agents rolled out, each one made a piece of his job faster. Eligibility drafts appeared in seconds. Document checks that used to take ten minutes took one. He liked all three individually, and cleared each queue without much thought about the others.

A colleague on Rowan's team cleared her document queue one Thursday afternoon, approving a claim's paperwork because everything on her screen checked out. She had no way of knowing, from that screen, that the fraud agent had flagged the same claim in a different tab an hour earlier, one she hadn't gotten to yet.

Three agents' calls, one screen. Nothing new is being computed, only shown together.

A routine county audit weeks later caught the mismatch, a genuinely fraudulent claim that had been fully approved, one piece at a time, by two different queues that never once talked to each other. Nobody had done anything wrong. The screens themselves made the miss almost inevitable.

Both screens show everything the agents know. Only one of them is usable at real volume.

With the redesigned view, that same claim appears once, as one entry, with the fraud flag and the document status sitting side by side, auto-expanded the moment the two disagree. Rowan's colleague sees both signals in the same glance she used to give only the document status, and the claim gets escalated instead of quietly approved.

The tempting shortcut was the one we didn't take.

I let each team ship its own screen because that was the fastest way to get three agents live. It took a routine audit, and a claim two people had each partly seen, to notice that "each screen works" and "the whole system works" were never the same claim.

SPARK, in one screenNot a lecture on multi-agent dashboards. SPARK is what tells you whose head the context lives in.

The recap, one line per letter: situation is three agents in three disconnected screens, payoff is one decision per claim instead of three, anchor is a claim-centered view that collapses agreement and expands disagreement, risk is burying the common case under too much default detail, and keep out is refusing to auto-blend a disagreement into a false single number.

And if you want to be sure it really works, try it somewhere elseSame five letters, a factory floor instead of a benefits office.

Ashgrove Precision Manufacturing runs three AI quality checks on every finished part: a visual defect scan, a dimensional-tolerance check, and a packaging-compliance check. Ludmila Petrova supervises all three from the line's quality station.

Mapped onto SPARK: situation is the same shape, three separate agent screens for the same part; payoff is Ludmila approving or holding one part with all three checks visible together, instead of clearing three separate pass or fail lists; anchor is the identical fix, a part-centered view, collapsed when all three checks agree, expanded the moment any two disagree; risk is the same, full detail on every part buries the rare real conflict under the volume of parts that pass everything; keep out is the same refusal, never auto-blend a visual flag and a tolerance miss into one score, since a person needs to see which check actually failed and why.

Same logic as the benefits office. The upper left corner is where a person's real attention belongs.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "build one screen around the claim, not three around the agents, collapsed unless they disagree," and stop.
Cost: there's no budget to rebuild all three teams' dashboards this quarter. Say so honestly, and start with a thin layer that just flags cross-agent disagreement on top of the existing three queues, since even a flag alone closes most of the gap.
The model gets better, for real: if all three agents' accuracy improves, disagreements between them get rarer, which makes the collapsed default view even more valuable, since there's even less reason to show full detail on the common case.

Where people run it wrong.
They let each agent ship with its own dashboard because that's the fastest way to get each one live, and never revisit it once all three are running together.
They average or blend multiple agents' scores into one number, hiding the specific thing they actually disagree about.
They build the full multi-agent detail view for every case, burying the rare real conflict under the volume of routine agreement.

How to use it live. When someone asks how to design for supervising multiple agents, ask one question first: if two of them disagreed on the same case right now, would anyone actually see it? If the honest answer is no, that's the whole design problem, before anything else.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0