Explain the compliance implications of an agent that acts autonomously.

Copperline Ledger is expense-management software. Its newest feature is an agent that reads a submitted expense, decides whether it fits policy, and pays it, without waiting on a person. Ines Kowalski is the controller who signed off on turning it on for her finance team.

How to answer this, stage by stage

Eight stages. The question sounds abstract, so the first job is making it concrete fast.

Let's learn

Say a finance team gives an agent the power to read an expense report, decide if it fits policy, and pay it, on its own, without anyone signing off first.

At launch, the agent proposed a payment and a controller reviewed every single one before it went out, a pause of maybe ninety seconds per expense. The team called this "assisted," and it worked well for months.

Then the team removed the pause for anything under a set dollar threshold, to save the controller's time. That single change is the whole story. The controller's job didn't get a little lighter. It changed shape entirely, from reviewing every payment to reviewing almost none of them.

At its worst: a vendor record that had existed for eleven minutes gets an invoice approved and paid the same afternoon, purely because the amount sat just under the review threshold. Nobody catches it until reconciliation, three weeks later, and only because someone happened to notice the vendor's name looked slightly off.

What I would leave alone: the small, frequent, easily reversed cases, a coffee run, a taxi receipt, genuinely don't need a human pause. Rolling review back in for those would slow the whole system down for almost no real protection.

The lesson: a threshold that quietly decides who reviews what is a compliance decision, whether anyone labels it that way or not.

Now here is the same thing as a story

The short version above is what you'd say defending this design. Read this one for how Ines actually found the gap.

Ines Kowalski has run Copperline's finance operations for five years. Before the agent, she reviewed every single expense herself, a habit she'd built over hundreds of small, unglamorous approvals a week.

For the agent's first months, it only proposed a decision, and Ines reviewed every one before it paid, ninety seconds at a time. She trusted it because it kept being right, and the review felt more like a formality than real work.

Five steps, and the third one was the only one that ever asked a person a question.

Then the team turned off that pause for anything under five hundred dollars, to free up Ines's time for bigger judgment calls. Over about six weeks, that quietly became almost all of it.

Nobody decided to stop reviewing. Bulk approval was just what "keeping up" looked like by month nine.

Then, in month eleven, a colleague on a neighboring team mentioned, almost in passing, that her own team's agent had nearly paid a duplicate wire before someone caught it in reconciliation. Ines pulled her own team's log that afternoon.

Two settings, and nothing in between. Either a person looked first, or nobody did.

What she found: an invoice from a vendor record created eleven minutes earlier, approved and paid the same afternoon, sitting quietly under the five-hundred-dollar threshold. It turned out to be a legitimate new supplier. It could just as easily not have been, and nothing in the system would have looked any different either way.

We designed for a dial the controller could turn. What she actually had was a switch.

Ines sorted every action type the agent could take by two questions: how often does it happen, and how hard is it to undo.

The upper-left corner, rare and hard to undo, is exactly where the pause had quietly gone missing.

With the redesign, any new-vendor payment or anything above a much smaller, risk-based threshold gets a mandatory human look, regardless of dollar amount, while routine small expenses stay fully autonomous. The same near-miss invoice now stops for review before it pays, in about the same ninety seconds it always used to take.

The old design asked how fast the agent could go. The new one also asks which of its actions a person genuinely can't afford to miss.

I merged propose-and-pay into one step because ninety seconds felt like a small thing to save at the time. It took a colleague's near miss, not my own, to see that the threshold deciding who gets that ninety seconds was a compliance decision all along.

The five steps, if you want to remember itFLIPS on an agent instead of a screen. The flip isn't the agent's behavior, it's what the controller stopped doing.

The five letters, kept in view. The middle one, I, is the only hard step in the whole method.

The recap, one line per letter: find the person is Ines and her five years of hands-on review, locate the habit is the ninety-second pause fading over six weeks, identify the flip is reviewing everything versus reviewing almost nothing, pinpoint the old decision is merging propose-and-pay under one threshold, and show the replay is the same near-miss invoice now stopping before it pays.

And if you want to be sure it really works, try it somewhere elseA different flip family, a custom manufacturing shop instead of a finance team. This time the agent isn't paying money, it's accepting orders.

Anvilcraft Manufacturing runs a quoting agent that reads incoming custom-part requests, checks them against shop capacity and material cost, and accepts the order on its own. Tobias Amara manages that quoting system.

A different flip here: not delegation, but scope. Tobias's team used to review a full daily sample of every accepted quote against the shop's actual capacity. As order volume tripled after a slow rollout to more customers, they narrowed that review to a small slice each day, a handful of quotes instead of the whole batch. The unreviewed slice is exactly where the agent started quietly over-committing the shop's welding capacity for a specific alloy, since that constraint had never come up often enough to be in its training examples. Nobody noticed until a customer's delivery slipped by three weeks and the shop discovered four other orders competing for the same machine time. The old decision: the daily review sample was fixed at the same size regardless of how much total volume it was meant to represent. The fix: scale the review slice with volume, and add a mandatory pause specifically for any order requiring a machine or material the agent has quoted fewer than a set number of times before.

A different flip family, a different shop floor, and the same upper-left corner still hiding the risk.

Swap the trigger and it still runs.
Speed: an interviewer caps you at a minute. Say "the risk isn't a wrong decision, it's the missing pause, so keep one for anything hard to reverse," and stop.
Cost: engineering says building a risk-based threshold is a whole quarter's work. Start with a single hard rule, any brand-new vendor gets a pause, no matter the amount, while the fuller system gets built.
The model gets better, for real: if the agent's accuracy keeps improving, that's still not a reason to remove the pause on hard-to-reverse actions. A more accurate agent that occasionally acts on a bad case with no review is still a bad case with no review.

Where people run it wrong.
They set the review threshold by dollar amount alone, when reversibility is the thing that actually matters.
They assume a low error rate means the pause is safe to remove everywhere, instead of just in the cases that are cheap to get wrong.
They re-certify the automation once, at launch, and never check whether it's still safe as volume and vendor variety grow.

How to use it live. When this question comes up, ask yourself one thing first: what pause did a person used to have, and where exactly did it go. Answer that before you say anything about the model itself.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0