How would you design transparency for an agent that took ten actions?

At 2:14am, Marshal detected an anomaly on a regional network segment and started fixing it. By 2:56am it had taken ten separate actions, on its own, before Dmitri Yefremov, the shift lead at Thorncliff Networks, had even finished his coffee.

How to answer this, stage by stage

Ten actions is a lot to unpack live. Don't try to explain each one; show that you know how to sort them fast.

Let's learn

Thorncliff Networks runs Marshal, an AI agent that watches the network and fixes outages on its own, without waiting for a human to approve each step.

Before Marshal, a network fault meant an engineer paged awake at 2am, running through the same ten-step playbook by hand, usually finishing somewhere around the hour mark once they were fully alert and had read the right logs.

Ten actions, one incident. Three of them, in red, are the ones that actually can't be quietly undone.

With Marshal, the same ten steps ran in 42 minutes, unattended, before Dmitri even opened his laptop.

The turn. The problem was never that Marshal took ten actions instead of one. The problem was that all ten got logged the exact same way, which meant the three that actually mattered were sitting in the same pile as the seven that didn't.

At its worst: a customer got a "service disrupted" text five minutes after their neighborhood's cell site had already gone dark, with no explanation of why, and Thorncliff's support line took the brunt of it the next morning.

What I would leave alone: detecting the anomaly and filing the incident ticket. Both are fully reversible, nobody has ever asked why either one happened, and giving them the same explanation weight as disabling a site would only slow down whoever's reviewing the incident.

Now here is the same thing as a story

The short version above is what you'd say to Thorncliff's ops leadership. Read this one for how the ranking actually got built.

Dmitri Yefremov has run the night shift at Thorncliff Networks for four years, and used to be the one running the ten-step playbook by hand before Marshal existed.

Marshal's first quarter live was quiet in a good way. Outages that used to take an hour of a groggy 2am fix now resolved in under an hour of Marshal working alone, often before Dmitri's laptop finished waking up.

At 2:14am on a Tuesday, Marshal caught a traffic anomaly on a regional segment. Over the next 42 minutes, it took ten actions in sequence, rerouting traffic, throttling bandwidth, restarting a router, escalating to Dmitri, adjusting the load balancer, disabling a faulty cell site, notifying customers, filing a ticket, and rolling back a bad config change.

Seven of the ten actions were fully reversible. Three weren't. The logging treated all ten the same.

Dmitri didn't notice a problem that night. Marshal's incident summary read clean: ten actions, outage resolved, all green. The trigger came three days later, from a customer service escalation: dozens of residents in one neighborhood had called in confused about a five-minute gap between their phones going dark and getting any explanation at all.

Dmitri pulled the full action log and, for the first time, actually plotted when each action happened against when complaint calls came in.

The team considered simply moving the notify step earlier in Marshal's sequence, and rejected it as the whole fix. Notifying before confirming which site was actually affected risked sending wrong information to the wrong neighborhood, a worse failure than a short delay.

Three actions cluster in the top right. Everything else sits safely in the bottom left, where a one-line log is plenty.

The redesign ranked all ten actions by how hard each was to undo, gave the top three full trace by default, shown right in the incident summary without anyone having to dig for it, and tightened the gap between disabling a site and notifying customers from five minutes to under ninety seconds.

Every one of the ten actions gets this record. Only the top three surface it by default instead of on request.

I built one flat logging format because Marshal only handled a couple of action types at launch, and a single format felt like the simpler, more maintainable choice. It took a neighborhood's worth of confused phone calls, over an outage that Marshal actually fixed correctly, to see that treating every action the same wasn't neutral. It buried the three explanations that mattered under seven that never got asked about.

ORDER, for ten actions at onceNot a story about a mistake. ORDER is what forces you to rank instead of explaining everything equally.

The recap, one line per letter: outcome is trustworthy, fast resolution, reversibility is the split between the three hard-to-undo actions and the seven easy ones, dependency is why notify has to wait for site confirmation, evidence is last quarter's actual questions asked, and rank is full trace for three, a log line for the rest.

And if you want to be sure it really works, try it somewhere elseSame five letters, a public library archive agent instead of a telecom network. A completely different field, and the hardest-to-undo action here isn't a customer-facing one at all.

Elsmere Public Library runs Pathkeeper, an AI agent that scans the digital archive each morning, merging duplicate records and flagging items for deaccessioning. Wren Talbot is the library's digital archivist, and reviews Pathkeeper's overnight actions each morning before the reading room opens.

Mapped onto ORDER: outcome is a clean, trustworthy archive that patrons and researchers can rely on. Reversibility is the split between merging two clearly identical duplicate scans, which is trivial to undo, and deaccessioning an item, permanently removing it from the collection, which is not. Dependency is that Pathkeeper can't deaccession anything until a human curator has reviewed the flag, a hard rule regardless of how confident the agent is. Evidence is a quarter of logs showing curators only ever ask to see the reasoning behind deaccession flags, never behind routine duplicate merges. Rank is full trace and mandatory human sign-off for every deaccession, a simple log for every merge.

The same tree that sorted Marshal's ten actions sorts Pathkeeper's, even though the two agents do nothing alike.

Only one milestone on this timeline needs a human before it can proceed. The rest just need to be logged.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "rank the ten by how hard each is to undo, full trace for the top few, a log line for the rest," and stop.
Cost: there's no budget for a full explanation-layer rebuild this quarter. Say so honestly, and ship the ranking and the notify-timing fix first; they're the cheapest changes and they catch the actual failure that happened.
The model gets better, for real: if Marshal's fix success rate climbs next year, the ranking doesn't change. A more accurate agent still needs its three hardest-to-undo actions explained, since accuracy and reversibility are different questions entirely.

Where people run it wrong.
They try to explain all ten actions equally, and end up with a wall of logs nobody reads in full.
They assume the last action taken is the most important one to explain, when it's often an early, hard-to-undo action that actually caused the damage.
They build the explanation layer around what's easy to log, not around what a person will actually go looking for later.

How to use it live. When someone hands you a list of actions an agent took, don't start explaining them in order. Ask first: which of these can't be quietly undone? Rank from there, and the design mostly writes itself.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0