What product decisions are unique to an agent versus a single-turn AI feature?

Cascade Grid hosts infrastructure for other companies. Foreman is the agent that watches its alerts and can act on them: restart a service, roll back a deploy, scale up replicas. Denny Okafor leads the site reliability team and carries the pager most weeks.

How to answer this, stage by stage

Six stages. This is a concept question, so the goal is naming the one decision that's genuinely new, not walking through a whole incident.

Let's learn

Foreman is an agent Cascade Grid built to watch infrastructure alerts and act on them: restart a stuck service, roll back a bad deploy, scale replicas up when traffic spikes.

Before Foreman, an alert paged whoever was on call. They read the logs, worked out the cause, and picked a fix. That took about 18 minutes on average, start to finish.

With Foreman acting directly, the same class of alert gets a fix moving in under 90 seconds. That's the number everyone quotes. It's also not the number that matters most.

The turn. The extra speed isn't the real story. The real story is a decision Cascade Grid never had to make for any single-turn tool: for each kind of action Foreman can take, does it run on its own, or does it wait for a person to say go? A log-summarizing chatbot never needed this decision, because it never does anything. Foreman does something every time it runs, and that "does something" is exactly where the new product decisions live.

What I would leave alone: restarting a single stateless pod that's already failing health checks. It's cheap, it's fast to undo, and gating it behind a confirmation would just slow down the easiest, safest case for no real benefit.

The lesson: a single-turn feature's worst case is a bad sentence someone reads and ignores. An agent's worst case is a bad action that already ran. That difference alone is the reason autonomy has to be its own product decision, built on top of the usual ones, not folded into them.

Now here is the same thing as a story

The short version above is what you'd say cold in an interview. Read this one for how Denny actually found the flaw in the confidence-based design.

Denny Okafor has run the SRE team at Cascade Grid for four years. Ask him which service breaks first under load and he'll name it before you finish the question.

Foreman shipped in the spring, and for months it was the best change the team had made in years. An alert fired, Foreman diagnosed it, and if its confidence crossed 90 percent it just acted. Denny watched the dashboard drop from 18-minute average incident time to under two minutes, and he genuinely loved it.

Four of these five steps existed for the old single-turn diagnostic tool too. The gate check is the one that's genuinely new.

Then, on a Wednesday, Foreman diagnosed a memory leak in a shared cache node with 94 percent confidence and restarted it. The diagnosis was right. The action was wrong anyway: that cache node fed six other services, and the restart dropped all six for eleven minutes while they reconnected, cascading into a customer-facing outage far bigger than the original leak would ever have caused on its own.

Both panels are "the agent did something." Only one of them is cheap to be wrong about.

Denny pulled the incident log and asked the question the team hadn't actually asked before: was 94 percent confidence ever supposed to answer "is this safe to undo," or had it only ever answered "is this diagnosis correct"? It was the second one. Nobody had built a separate signal for the first.

The cache restart sits in the worst corner: wide reach, hard to undo. Confidence never measured either axis.

Foreman's autonomy rule got rebuilt around reach and reversibility instead. Restarting a single pod, narrow reach, easy to undo, still runs alone. Restarting anything more than one dependent service now waits for a one-click confirmation, regardless of how confident the diagnosis is.

Only one branch out of four still runs unattended. That's on purpose.

The same Wednesday, replayed under the new rule: Foreman still diagnoses the leak in seconds, still proposes the restart, but the shared cache node trips the reach check and a confirmation lands on Denny's screen instead of the action just happening. He approves it in four seconds, because the diagnosis actually was right, and the restart happens in a maintenance window instead of live, with the six dependent services warned first. Zero customer-facing outage instead of eleven minutes across six services.

The old rule asked "how sure is the model." The new one asks "how bad is it if the model's wrong anyway." I built the first version because confidence was a number we already had sitting right there, and it felt wasteful not to use it. It took one cascading outage to see that having a number isn't the same as having the right number.

PICK, laid out for an agent's autonomy lineFour letters. The C step is the one that actually does the work here.

The recap, one line per letter: position is "ask first by default for anything hard to undo," impact is a few seconds against an hour, cost asymmetry is cheap-and-constant versus rare-and-huge, and kill criteria is watching the unread-approval rate for the moment the gate stops working.

Confidence is real information. It's just fourth on the list, not first.

And if you want to be sure it really works, try it somewhere elseSame four letters, an irrigation valve instead of a cache node. A completely different field, and the reach axis turns into water instead of dependent services.

Thistlewood Farms runs a field-irrigation agent that reads soil-moisture sensors and can open or close valves on its own. Rosalind Petit manages the system across nine hundred acres.

Mapped onto PICK: position is that any valve action affecting more than one field waits for Rosalind's approval, while a single-field top-up runs on its own. Impact is that waiting costs her a text message and thirty seconds; acting alone and being wrong on a shared header valve can flood a neighboring field mid-harvest, ruining a week of drying time for the crop already cut and lying in the sun. Cost asymmetry is the same shape as Foreman's: the cheap error happens constantly and is shrugged off, the expensive one happens rarely and wipes out a harvest. Kill criteria here is different from Cascade Grid's: if Rosalind starts approving valve requests from her truck without checking the field map first, the confirmation isn't protecting anything anymore, and the fix is a photo of the field attached to the request, not more approvals.

Reach, on a farm, means rows flooded, not services restarted. The shape of the decision is identical.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "the new decision is where the ask-first line sits, set by reach and reversibility, not confidence," and stop.
Cost: there's no budget to build a real reach-scoring system this quarter. Say so honestly, and start with a manual list of three known wide-reach action types that always require confirmation, since even a rough list beats none.
The model gets better, for real: if Foreman's diagnosis accuracy climbs to 99 percent, that's still not a reason to loosen the reach-based gate, since a rarer wrong diagnosis on a wide-reach action is still exactly as expensive as before.

Where people run it wrong.
They gate every action the same way regardless of reach, which slows the agent down until it's not worth building.
They use the model's own confidence as the autonomy signal, which answers a different question than the one that matters.
They build the gate once and never watch whether people still read what it shows them.

How to use it live. If you get stuck, ask yourself one question: what does this feature actually do to the world, not just say to a person? If the honest answer is "nothing, it just displays text," you're looking at a single-turn feature. If the answer is "it changes something," you've found the agent-specific decision, and it's always about where that action sits between "runs alone" and "waits."

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0