CaseAdvancedEval-Driven Specification / Writing a PRD for an AI feature / #24

How would you write a PRD for an agent that takes actions on a user's behalf?

The direct answer
Write the PRD so autonomy is a property of the action, not the agent. Let it keep repeating an exact reorder on its own, but any action that changes what a person actually gets, a different brand, a different size, a swap on an item tied to a diet or an allergy, has to wait for one explicit yes from the person it affects before it ships. Then track how often a substitution gets kept without that yes, because that is the one number that tells you it is happening before someone's bank statement does.
Do this, in order
  1. Require an explicit yes before any substituted item ships a second time, not a card charge going through.Why: this is the one decision the whole risk in this PRD turns on.
  2. Flag items tied to a diet or allergy so a substitution on them needs a yes the very first time, not just the second.Why: this is where a wrong order costs the most, not where it happens most often.
  3. Send that confirmation to whoever's tagged as the household's shopper, not only the account holder's inbox.Why: the person who eats the mistake almost never opens the order history.
  4. Keep every exact-match repeat reorder fully automatic, no confirmation, no delay.Why: this is the whole point of the agent, and nothing here is actually at risk.
  5. Log the real reason an order came back different, not just fulfilled or not.Why: a swap that isn't logged as a swap is invisible to everyone, including the team that built it.
  6. Track how many diet-flagged items get re-substituted with no confirmation, and alert on the second one.Why: this is how the team finds out before a customer does.

How to answer this, stage by stage

Seven moves. The middle three, splitting who authorized the agent from who it acts on, naming the action that can't be undone, and writing the actual PRD line, are where the real answer lives.

1
Ground it in one real product before naming a framework
Say it like this
"Say we build Restock. It watches a household's shared shopping list, kept in sync through an app, and reorders groceries on its own before they run out. It charges a card already on file and ships through a delivery partner. The question the PRD has to answer is which of its actions get to happen alone, and which ones need a person to say yes first."
Why this works
Grounds a PRD question in one real product before naming a method, so the answer can't stay abstract.
2
Say your structure out loud
Say it like this
"I'd use GUARD, because a question about what an agent's allowed to do on its own is really a risk question wearing a spec question's clothes. Who's affected and who only authorized it, where it lands worst, who can't tell it happened, the actual rule, and how you'd catch it going wrong."
Why this works
Two seconds that show a plan exists before the interviewer hears you improvise a list of features.
3
Split who authorized it from who eats the mistake
Say it like this
"There's Theo. He set up the account, linked his card, and gets every notification the app sends. And there's his mother Aino, who lives with them, does the actual cooking, and has never opened the app once. Theo authorized Restock. Aino is who it acts on."
Why this works
This is GUARD's G step, naming both sides before proposing anything.
4
Point at the action that's expensive to take back
Say it like this
"Most of what Restock does is cheap to get wrong. Order the wrong roll of paper towels, you use two instead of one, nobody's hurt. But swap the brand of soup on someone managing her sodium, and keep swapping it quietly for six weeks, and there's no undo button. You can refund a charge. You can't unswallow six weeks of soup."
Why this works
This is U. It shows harm doesn't land evenly, using one concrete example instead of a general warning.
5
Name who can't tell it happened
Say it like this
"Aino never sees a receipt, a push alert, or an order log. Theo only found the pattern because he was scanning his card statement one evening and noticed the same delivery line item, six weeks running. If he hadn't looked, nobody in that house would know why the soup started tasting different."
Why this works
This is A, GUARD's hardest step, and the one most PRDs skip.
6
Write the actual PRD line, not a policy line
Say it like this
"Here's the line I'd put in the doc. Any reorder where the brand, size, or item differs from the household's last order needs one explicit yes from the household's shopper before it ships, sent as a real alert, not logged where nobody looks. Items tagged for a diet or allergy need that yes every time a substitute shows up, not just the first. An exact repeat of the same item, same size, ships with no confirmation at all, same as today."
Why this works
This is R, and it's the actual answer. It names one class of action, not a blanket rule for everything.
7
Say how you'd catch it in production, then close
Say it like this
"I'd track how often a substituted item gets reordered a second time with no confirmation logged, broken out by whether the item's diet-flagged, and alert the team the moment that happens twice for one household. That's the whole answer. Let the boring reorders stay silent, put a real yes in front of the ones that change what somebody eats, and build a way to catch it if that rule ever gets skipped."
Why this works
This is D, plus a close that restates the decision in one breath.

Let's learn

What happens when the agent that's supposed to save a household time quietly decides, on its own, what somebody eats?

Say we build Restock. It reads a household's shared shopping list and reorders groceries before they run out, charging a card on file and shipping through a delivery partner.

Before Restock, Theo Sundqvist spent about 25 minutes most weeks going through that list by hand: checking what was low, opening the grocery app, picking the right size and brand, placing the order himself.

With Restock watching the same list, that's down to about 3 minutes a week, mostly just glancing at a text saying the order went out. It gets the exact item right, same brand, same size as last time, 96 times out of 100.

Knowledge spark: what does "fulfilled" actually mean? A status a delivery partner sends back once a box has shipped. It means the order left the warehouse. It says nothing about whether it's the same brand, the same size, or the right thing for whoever's about to eat it.

The other 4 times out of 100, the usual item is out of stock, so Restock accepts whatever the delivery partner substitutes. That part isn't the problem. Stock runs out, that's normal. The problem is what Restock does the next time that same item runs low. The delivery partner marks a substituted order fulfilled, same as any other order, and Restock reads fulfilled as proof the swap was fine. So it orders the substitute again. And again, with nobody ever asked.

We didn't build an agent that gets a household's order wrong. We built one that quietly changes what somebody eats and never says so.

At its worst, this lands on whoever in the house didn't set the agent up. A brand of soup changes because of one out-of-stock Tuesday, and six weeks later it's still the wrong brand, quietly reordered five more times, eaten by someone managing a low-sodium diet who never chose it and never knew it changed.

The decision I would take back We let a delivery partner's "fulfilled" status count as approval to keep reordering a substitute. That was a fine shortcut when substitutions were rare and small. It stopped being fine the moment a swap could sit on a diet flag for weeks with nobody asked.

What I would leave alone. Every exact repeat, same brand, same size, same item bought a dozen times before, needs nothing. No confirmation, no delay. That's most of what Restock does, and none of it is the risk.

The lesson. An agent that's fine acting alone nine times out of ten will still get treated like it's fine acting alone the tenth time, unless the PRD names the exception. "Use good judgment on sensitive items" isn't a requirement anyone can build against.

Now here is the same thing as a story

Read the short version above if you want it fast. Read this one when you want to feel why the rule matters, not just know what it is.

Noora Kettunen has run product for Restock's ordering pipeline since the private beta, thirty households that first spring, mostly friends of friends willing to put a card on file for an agent that wasn't finished yet.

For those first two months, she read every substitution herself. Ninety of them came through in that window, and she opened each one: what ran out, what shipped instead, whether it was close enough that nobody would notice or different enough that somebody should be asked. Two of the ninety, both on items a household had flagged for a health reason, she pulled and sent back for a person to call before they ever repeated.

Then the beta grew. A hundred households by early summer. Six hundred by the end of it. By the time Restock opened to four thousand households that fall, Noora had stopped opening substitutions one by one. She trusted the delivery partner's fulfilled status the same way the system did, and the weekly dashboard kept saying orders complete, on time, no complaints. Complete started to feel the same as correct.

Nobody told her this was a problem. Every week, the numbers said it was working.

Then a note came in from support, not urgent, just odd: a household asking why their soup delivery "wasn't the one we usually get," six weeks running, when they'd never changed anything on their end.

Noora pulled the order. Theo Sundqvist's account. The usual low-sodium tomato soup had gone out of stock back in week one; the delivery partner substituted the regular version; Restock logged it fulfilled and reordered the same substitute five more times since, once a week, right on schedule. Theo hadn't caught it on the app, he barely opened it. He'd found the pattern going through his card statement, the same line item six times, and finally looked closer.

His mother Aino does the actual cooking. She'd been making soup from whatever showed up, the way she always had, and had mentioned to Theo that it tasted saltier than she remembered. Neither of them had connected it to the app.

Two figures side by side. Theo, on the left, is labeled as the one who sets the reorder rule and gets every alert. Aino, on the right, is labeled as the one who cooks from the list and sees no alert at all.
Same soup can, same kitchen. Only one of them ever gets asked.
We didn't build an agent that gets a household's order wrong. We built one that quietly changes what somebody eats and never says so.

That weekend Noora pulled two samples to see how big the gap actually was. Three hundred forty diet-flagged substitution events from the last quarter, the ones nobody had reviewed by hand: forty-one of them, about 12 out of every 100, had repeated a second time with no confirmation ever logged. Then she pulled her own numbers from the first two months, when she'd read every one herself: two out of ninety, about 2 out of every 100. The gap wasn't the delivery partner failing more often. It was the review disappearing.

I would take back letting "fulfilled" stand in for a person's yes. Not the idea of substituting at all, that idea was right, a household would rather get the regular soup this week than no soup. The moment we let a shipping status decide whether to keep repeating that swap, we also promised ourselves we'd keep watching it by hand, and we stopped keeping that promise the day the beta passed a few hundred households.

Run the same year again with one line added to the PRD: any substitution on a diet-flagged item needs a yes from Aino, sent to her, not to Theo's inbox, before it ships a second time. The first swap still goes out, stock runs out, that's real. The second one waits. The soup never makes it to a sixth week.

The thing I'd tell myself, back when I wrote the first version of this pipeline: a shipping status was never a stand-in for asking someone. It just looked like one for as long as nobody checked.

Where the five letters of GUARD land here

This is a risk question, so the framework is GUARD. "Write a PRD for an agent that acts on someone's behalf" reads like a spec question, which is exactly why a shipping status standing in for consent didn't look like the real decision until a support note found it.

G, groups. The household member who set up Restock and linked a card. And the household members who actually eat what it orders, who may never open the app at all.
U, unequal. An exact repeat is nearly impossible to get wrong. The substitutions are where the risk lives, and the ones tied to a diet or an allergy stay wrong the longest, because nothing about a fulfilled order ever forces a second look.
Diet-flagged substitutions that repeated with no one asking
Two samples pulled the same weekend, before the PRD change shipped.
Reviewed by hand, first two months (90 entries)
2%
Auto-accepted at scale (340 entries)
12%
Forty-one of three hundred forty, against two of ninety. The weekly dashboard's "fulfilled" count never showed this, because it never asked what happened after fulfilled.
A, ability to contest. Aino authorized nothing and sees nothing: no receipt, no push alert, no order history she'd think to open. The only reason anyone caught it was Theo's own card statement, and that was luck, not design.
A flow from the soup running low, through a brand swap and being marked fulfilled, to a missing step asking Aino first, ending in the same swap shipping again.
The step that should sit before the swap repeats, and doesn't.
R, reduce. Any reorder where the brand, size, or item differs from the household's last order needs an explicit yes before it ships again, sent to whoever's tagged as the shopper on that item. Diet-flagged items need that yes every time a substitute shows up, not just the first. An exact repeat needs nothing.
D, detect. Track how often a diet-flagged substitution repeats with no confirmation logged, and alert the moment it happens twice for the same household. A weekly count that includes every fulfilled order, not just the ones that came back the way they went out.
Where this answer would fail If the fix is a longer terms-of-service line about substitutions, or telling the model to "use good judgment" on sensitive items, none of it counts. A confirmation step tied to a specific field, diet-flagged plus brand-changed, and a repeat-substitution alert are build items with an owner and a cost. Somebody can put them on a roadmap this quarter, and you can check whether they shipped.

And if you want to be sure it really works, try it somewhere else

A veterinary clinic runs an agent that auto-refills medication for pets on long-term prescriptions, sending it straight to a compounding pharmacy and telling the owner only that a package is on its way. Different industry entirely, same five letters, same gap.

G, groups. The clinic's front desk, who turned on auto-refill for convenience. And the pet owner, who picks up a refill assuming it's the exact formulation the vet wrote, because nobody told them to check.
U, unequal. A refill of the same tablet, same dose, is close to risk-free. A refill where the compounding pharmacy changed the concentration or swapped a filler ingredient, common when a preferred supplier is out, is the one that matters for a pet on a tight dose, like a thyroid or heart medication.
A, ability to contest. Curbside pickup means the owner never talks to anyone who'd mention a change. They assume refill means identical to last time, because nothing has ever told them otherwise.
R, reduce. Any refill where the compounding source, concentration, or formulation differs from the last fill needs a confirmation text to the owner before it ships, every time, on any chronic medication. A same-formulation refill needs nothing.
D, detect. Track repeat refills on chronic-medication prescriptions where the formulation changed and no confirmation was sent, and flag it before the next refill goes out on its own.

Swap the trigger and it still runs

  • Speed: the delivery partner doubles how many categories it's allowed to auto-substitute to hit a faster fulfillment number, and the confirmation step is the only thing standing between that speed and a bigger version of the same silent swap.
  • Cost: a cheaper compounding pharmacy comes on board and substitution volume triples because it's genuinely a better deal, right up until it hits a dose that matters.
  • The model gets better: Restock's out-of-stock prediction gets sharp enough to reorder three days earlier than before, and that's exactly the moment nobody wants to slow the pipeline down to ask a question, because the top-line number already looks finished.

Where people run it wrong

  • Writing "ask before major changes" once, in a policy document, with no PRD line naming the actual action.
  • Treating a shipping status as proof the order was right, when it only ever meant the box left the warehouse.
  • Checking for the confirmation gap once, at launch, and never again when a new delivery partner or item category gets added.

How to use it live

Ask, out loud, "would the person eating this notice if it changed?" If the honest answer is yes and nobody's asking them first, the rule isn't written yet. Say that, then name the one specific action in whatever example you're given that should have needed a yes and didn't.

Flashcards (click a card to flip it)

1 · THE FRAMEWORK
Which framework fits a PRD question about an agent that takes actions on a user's behalf, and why?
Tap to flip
ANSWER
GUARD, for risk. The real question isn't how to spec the actions, it's who authorized the agent versus who it acts on, where a wrong action lands hardest, and who never finds out, exactly what GUARD is built to find.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Noora Kettunen, the product manager who owns Restock's ordering pipeline. She used to personally review every substitution before it was allowed to repeat.
3 · THE HABIT
What did Noora stop doing once the beta grew?
Tap to flip
ANSWER
Reading every substitution herself before letting it repeat. Once the beta passed a few hundred households, she started trusting the delivery partner's "fulfilled" status the same way the system did.
4 · THE GAP
What's the two-number gap this whole answer turns on?
Tap to flip
ANSWER
About 2 of every 100 hand-reviewed diet-flagged substitutions would have quietly repeated as a silent default, against about 12 of every 100 once nobody was reviewing them.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Treating the delivery partner's "fulfilled" status as consent to keep reordering a substituted item. It made sense when Noora reviewed every one by hand, not once volume outran her.
6 · THE NUMBER
Fill in: Aino ate the wrong soup for ______ before Theo caught the repeat charge on his card statement.
Tap to flip
ANSWER
Six weeks. He found it doing a routine statement check, not because Restock ever told him.
7 · THE REPLAY
Same household, new PRD rule in place. What changes?
Tap to flip
ANSWER
The first substitution still ships, stock ran out, that's real. The second one waits for a yes from Aino, because the item's flagged for her diet. It never reaches a sixth silent week.
8 · TRANSFER
Section four runs GUARD again on a different product. Which one, and what does the reduce step become?
Tap to flip
ANSWER
A veterinary clinic's auto-refill pipeline. Reduce: any refill where the compounding source, concentration, or formulation differs from the last fill needs a confirmation text before pickup, every time, on chronic medication.

Check yourself Score: 0 / 0

Short answer
1. Why isn't a delivery partner's "fulfilled" status enough proof that a substituted grocery item was the right call?
Show hint
Think about what "fulfilled" actually measures: did a box ship, or was it the right box.
Show answer
Model answer: "Because fulfilled only means the order shipped. It says nothing about whether the brand, size, or item is the one the household actually wanted, or safe for whoever's eating it. A system built to check 'did it ship' can't also tell you 'was it right,' those are two different questions with two different answers."
Fill in the blank
2. In the backlog sample, ______ out of 340 auto-accepted diet-flagged substitutions repeated with no confirmation, against ______ out of 90 entries Noora had reviewed by hand.
Show hint
It's the pair of numbers behind "the gap" flashcard.
Show answer
41 out of 340, and 2 out of 90. About 12 percent against about 2 percent. The delivery partner marked every one of them fulfilled either way, because fulfilled was never built to catch a wrong brand.
True or false
3. True or false: linking a card and turning on Restock counts as authorizing every substitution the agent makes afterward, for as long as the account stays open.
  • True
  • False
Show hint
Ask what turning the agent on was actually agreeing to, and for how long that agreement should reasonably stretch.
Show answer
False. Turning the agent on authorized routine reordering of the items the household already buys. It was never asked, and never answered, whether it also covers a brand swap repeating silently for weeks on an item somebody else in the house eats.
Multiple choice
4. Which old decision does this answer take back?
  • A. Treating the delivery partner's "fulfilled" status as consent to keep reordering a substitute.
  • B. Letting Restock reorder groceries automatically at all.
  • C. Adding a confirmation step for diet-flagged substitutions.
  • D. Telling the delivery partner to ship faster.
Show hint
Look for the decision made when the pipeline was first built, not the fix proposed after.
Show answer
A. C is the fix, not the reversal. B gives up on the agent instead of designing something safer. D is unrelated, a dial nobody proposed turning. Only A names the actual choice, a shipping status standing in for a person's yes, that this answer undoes.
Multiple choice
5. Which of these actions is safe to leave fully automatic, with no confirmation step at all?
  • A. Reordering the exact same brand and size the household bought last time.
  • B. Reordering a different brand of an item tagged for a diet or allergy.
  • C. Repeating a substitute a second time on a diet-flagged item.
  • D. Switching to a larger size than the household has ever bought.
Show hint
Ask which one of these changes nothing about what actually shows up at the door.
Show answer
A. An exact repeat changes nothing, so there's nothing to confirm. B, C, and D all change what the household actually receives, exactly the class of action this PRD requires a yes for.
Short answer, apply it yourself
6. Pick a product you use yourself that's allowed to act on your behalf without asking each time. What's one class of action inside it that quietly needs a confirm step it doesn't have?
Show hint
Look for the action that changes something you'd actually notice, not the ones that just repeat what you already agreed to.
Show answer
Model answer: "My bank app auto-pays a set of recurring bills. It's fine reordering the same payment every month. But if a biller's amount jumps by a lot more than usual, that's a different action wearing the same label, and it still goes through with no confirmation at all. That's the exact gap this answer argues for closing."
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more