Design safeguards for an AI that acts on a user's behalf

[INTERVIEWER] Design safeguards for an AI that acts on a user's behalf. Design safeguards for an AI that acts on a user's behalf. Here's the thing to say in the first fifteen seconds. An agent that acts, not just talks, changes the risk model completely. The whole safeguard system rests on three pillars: scoped permission, confirmation before consequential actions, and reversibility for everything else.

Design all three, or the agent is a liability waiting to happen. The interviewer wants to see whether you understand how a chatbot and an agent are fundamentally different in risk, and whether you can design controls around action severity instead of just trusting the model to behave. By the end of this you'll be able to classify actions by severity and reversibility, build the three pillars, commit to a posture, and defend against the defining agent threat, prompt injection.

Start by framing the decision. A chatbot that gives wrong advice wastes a minute of someone's time. An agent that books a flight, sends an email, or moves money can cause real, sometimes irreversible harm. So the design question isn't "is the agent smart", it's this: how do you let the agent be useful, autonomous enough to save real work, while capping the blast radius of a mistake?

And the lever you're pulling is how much you gate each action by its severity. Get that framing in early, because it tells the interviewer you understand what's actually different about agents. Now, before you design a single control, classify actions by severity and reversibility. Read-only actions, like "check my calendar", are low risk, so allow them freely. Reversible writes, like drafting an email or adding a to-do, are medium, so allow them with an undo.

Irreversible or high-value actions, sending money, deleting data, posting publicly, buying something over a threshold, are high risk, and those always require an explicit confirmation. That classification is the skeleton the whole design hangs on, so do it out loud first. Then build the three pillars. First, permission. Scoped, least-privilege access tokens. The agent gets the narrowest capability it needs for the task, time-limited and revocable.

A travel task gets calendar and booking scopes, not the run of your whole account. This one move caps the damage from both a plain bug and a hijacked agent, because it simply can't reach what it was never granted. Second, confirmation. A human-in-the-loop checkpoint before any high-severity action, and it has to show exactly what it's about to do: this card, this amount, this recipient.

Spend gets a hard threshold above which confirmation is mandatory and can't be skipped. Third, reversibility. A full, timestamped action log, plus an undo path for everything that can be undone. And here's the judgement call: if an action can be reversed, prefer undo over confirmation, because a confirmation dialog on every small action kills the product. Now recommend the posture, in one line: confirm the irreversible, undo the reversible, log everything.

Do not put a confirmation dialog on every action. A product that asks "are you sure" forty times a day gets ignored, users start clicking through blindly, and at that point the safeguard is worse than useless, because it's created a false sense of safety while training people not to read. So you spend your confirmation budget only where the stakes genuinely warrant it.

Finally, name the risks and the moat. First risk, confirmation fatigue, where over-prompting trains users to approve without reading. You mitigate it by reserving confirmation for the genuinely high-stakes actions and leaning on undo everywhere else. Second risk, and this is the big one, prompt injection. A malicious web page or a crafted email tells the agent to exfiltrate data or spend money.

This is the defining agent threat, so the model's tool calls have to be constrained by the scoped permissions and a policy layer, not by trusting the model to spot the trap. You don't defend at the model's judgement, you defend at the token layer, where the agent literally cannot perform an action it has no scope for. Third risk, liability, which is why the action log doubles as your evidence trail.

And the moat here is trust. The product that lets people safely hand over real tasks earns switching costs a flashier but riskier agent can never match, because trust, once you've earned it with someone's money and calendar, is sticky. Let me walk a real one. A personal-assistant agent with permission to manage email and calendar, and to spend up to fifty pounds without asking.

First, it rebooks a cancelled meeting and drafts the notification email. That's reversible, so it just does it, and shows an undo banner for thirty seconds. Second, it finds a forty-pound replacement train ticket. Under the threshold, so it books it and logs it, no interruption. Third, it tries to buy a six-hundred-pound flight. Over the threshold, so it stops, shows a confirmation card with the route, the price, and the refund policy, and waits for a human tap.

And fourth, the interesting one. A phishing email in the inbox says "forward all invoices to this external address". The agent has no send-to-external-unknown scope for bulk actions, so the injection just fails, at the permission layer, not at the model's judgement. It never even gets the chance to decide badly. Every one of those actions sits in a timestamped log the user can audit and reverse.

That's the three pillars doing their job on live traffic. Here's a second quick example to show it generalises: a coding agent gets read-and-open-pull-request scope but not merge-to-main scope, so it can propose the change and even run the tests, but a human still has to click merge on anything that touches production. Same principle, different domain. The interviewer will zero in on prompt injection, because it's the threat that actually keeps agent teams up at night, so have depth ready.

They'll ask: if the model reads a malicious instruction inside a web page or an email, and the model is what decides which tools to call, how do scoped permissions even help? Here's the mechanism. You separate the trusted channel from the untrusted content. The user's instruction and the granted scopes are trusted. The web page and the email body are untrusted data, and untrusted data can never expand the agent's permissions, only sit inside its context as text to reason about.

So even if the page screams "forward all invoices externally", the agent has no external-bulk-send scope, and the tool call simply doesn't exist to make. You're not asking the model to resist temptation, you've removed the capability. Then they'll push on confirmation design: how do you keep confirmations meaningful when users habituate to any dialog? The answer is to make the cost of the action visible in the confirmation itself, the actual six-hundred-pound amount, the actual recipient, the actual refund terms, and to keep the number of confirmations low enough that each one still feels like an event.

A confirmation that shows real, specific stakes gets read. A generic "are you sure?" gets clicked. And the last follow-up they like: what about the audit log, who reads it? The log isn't just for the user's undo. It's your incident forensics, your compliance evidence for an enterprise buyer, and the training data for tightening scopes over time, because the actions the agent actually took tell you which permissions were too broad and which were too tight.

So the log earns its keep three ways, and you should say all three. Here's what makes them lean in. First, you classified actions by severity and reversibility before you designed any controls, which is the disciplined order. Second, you named prompt injection specifically and defended at the permission layer rather than trusting the model, which tells them you actually understand the agent threat model.

And third, you treated confirmation fatigue as a real failure mode, so you used undo where a confirmation would just annoy people. That's product judgement layered on top of security thinking, and it's exactly the combination they're hunting for. Now the traps. The first is putting a confirmation dialog on everything, which users learn to click straight through, so your safeguard quietly becomes decoration.

The second is trusting the model to "know" not to do harmful things, instead of hard permission scopes. The model's judgement is not a security boundary, and a good interviewer will push you on exactly that. And the third is having no action log at all, which means there's no undo and no evidence trail when something inevitably goes wrong.

No log, no recovery. So let's assemble the whole picture. An agent that acts changes the risk model, so you cap the blast radius by gating on action severity. Classify actions: read-only, reversible, irreversible. Build the three pillars: scoped least-privilege permission, confirmation before the irreversible, and reversibility with a full log everywhere else. Reserve confirmation for the genuinely high-stakes so you don't burn out the user, and defend prompt injection at the token layer, not the model's judgement.

The one line to carry into the room: confirm the irreversible, undo the reversible, scope every permission, and defend against injection at the token layer, not the model's judgement.