Describe how you would run a limited pilot for an agent with real-world side effects.

Interviewer's question: "Describe how you would run a limited pilot for an agent with real-world side effects." Marrowbrook Grocers is piloting Lucent Restock, an agent that drafts and can submit real purchase orders to suppliers. Torrance Feldspar runs the pilot. Junot Villanueva manages the one store where it's live.

How to answer this, stage by stage

Nobody is grading whether you can name a pilot's rollout percentage. They are grading whether you can say who gets hurt if the pilot goes wrong, and what stops it before it does.

Let's learn

Lucent Restock reads a store's shelf and warehouse data, predicts demand, and drafts real purchase orders to suppliers, submitting them without a person clicking send.

Before Lucent, Marrowbrook's store-level inventory manager spent about five hours a week manually reordering. Running out of a fast-moving item cost the store roughly six hundred dollars a week in lost sales, most weeks, on something.

Now the agent drafts a full reorder plan for the store in minutes, and most days it submits it without anyone reviewing it line by line.

Here is the turn: the risk here was never really "the agent orders the wrong thing." Ordering ten extra cases of a slow mover is a cheap mistake, caught fast, fixed next week. The real risk is an order, or a cancelled order, that nobody meant to happen, landing on someone with no way to see it coming and no way to push back, specifically a small supplier who isn't watching a dashboard the way Marrowbrook is.

One of them can stop the pilot in a sentence. The other one only finds out after the truck doesn't show up.

At its worst: a demand model quietly zeroes out a standing weekly order with a small local jam producer, during a produce oversupply cycle it read as a permanent shift. Nobody at Marrowbrook notices for weeks, since the store's own numbers still look fine. The jam producer notices immediately, because a truck that has come every Tuesday for two years simply stops.

What I would leave alone: routine, small quantity increases on fast-moving items that are already in the normal reorder cycle don't need a second look. Flagging every ordinary reorder would just rebuild the five hours a week the agent was supposed to remove.

The lesson: a pilot with real-world side effects doesn't earn the right to widen by being usually right. It earns it by proving the one thing it can't take back, it didn't do, not even once.

Now here is the same thing as a story

The short version above is what you'd say defending this pilot design to Marrowbrook's operations committee. Read this one for how the near miss actually got caught.

Torrance Feldspar had run the Lucent Restock pilot at one Marrowbrook store for three weeks, and it was going about as well as a pilot can go: fewer stockouts, a store manager who stopped dreading Monday mornings, no complaints.

Week three is the only week that decided whether weeks four through six were allowed to happen at all.

In week three, a regional produce oversupply pushed prices down across the board, and Lucent's demand model read the pattern as a permanent shift for one item: a small-batch preserves line the store had carried, slowly but steadily, for two years, supplied every week by a single grower an hour outside town.

The agent drafted a full cancellation of that standing weekly order. Under the pilot's original design, that draft would have gone out automatically, the same way every other reorder had for three weeks.

Three ordinary steps, and then a fourth one that, before that week, simply did not exist.

It didn't go out, because two weeks earlier, after a smaller near miss on a quantity spike, Torrance's team had already tightened the pilot's allowlist to block any action that cancelled a standing order, routing it to a person instead. The cancellation sat in a daily diff report Torrance reviewed every morning with coffee, flagged in red, waiting.

Torrance called the grower's own farm stand that afternoon, not to apologize for something that happened, but because nothing had, yet, and the order simply went out as usual that Tuesday. The grower never knew there had been a near miss at all.

The decision Torrance's team had made two weeks earlier, tightening the allowlist after a much smaller quantity-spike incident, was the only reason week three's near miss stayed a near miss. Before that, the pilot had trusted the model's judgment on ordinary reorders and assumed cancellations were just a smaller version of the same thing. They aren't. A quantity change is a dial. A cancelled standing order is a door somebody can't see closing.

Replayed under the original, untightened design: the cancellation goes out automatically on a Friday, the grower's truck simply doesn't get scheduled for the following Tuesday, and by the time anyone at Marrowbrook notices the numbers look slightly off, three weeks of a small farm's income are already gone, with nobody able to say why until someone finally calls to ask.

I believed, going into week one, that watching the aggregate numbers closely was the whole job of running a careful pilot. It took one grower who never even found out how close it came to see that the numbers I was watching would have looked completely normal the entire time.

GUARD, spelled out for one orderNot a compliance checklist. GUARD is what tells you why a hard allowlist beats a policy nobody enforces.

The first two items are ordinary automation. The last two are the actual guardrail.

Two of the three branches route straight to a person, on purpose, even though most orders never touch them.

The recap, one line per letter: groups is Torrance's team against small suppliers like the grower, unequal is who has no account rep to notice a cut order, ability to contest is that the grower has no way to see or appeal a demand model's decision, reduce is the hard allowlist that blocks standing-order cancellations, and detect is the daily diff that caught the near miss before it became a real one.

And if you want to be sure it really works, try it somewhere elseSame five letters, a home healthcare scheduling agent instead of a grocery supply chain. A different kind of side effect, the same shape of decision.

Fennsbury Home Care runs an agent that schedules in-home aide visits and can rebook or cancel a visit automatically when an aide calls in sick.

Mapped onto GUARD: groups are Harrow's scheduling coordinators, who can see the whole week's calendar, and homebound clients, who often can't easily reach anyone if a visit silently disappears. Unequal is that a client with a strong family support network probably calls someone the same day a visit is missed, while a client living alone may not notice until the missed visit itself becomes an emergency. Ability to contest is close to zero for that second client, no way to flag that a cancellation happened, no way to demand a replacement before real time passes. Reduce is the same shape of allowlist: the agent may reschedule a visit to a new time freely, but it may never cancel a visit outright without a person confirming a replacement is already booked. Detect is a same-day report of every cancelled visit, checked against whether a replacement aide was actually assigned, not just scheduled.

Swap the trigger and it still runs.
Speed: an interviewer caps you at thirty seconds. Say "hard allowlist, no cancellations without a person, daily diff before widening," and stop.
Cost: if building a real-time diff report is too expensive for a one-store pilot, ship a simpler end-of-day summary first, since catching a near miss a few hours late still beats not catching it at all.
The model gets better, for real: if the demand model's forecasts get much more accurate, that's still not a reason to let it cancel standing orders alone, since the group that can't contest a decision hasn't changed just because the model got better at making one.

Where people run it wrong.
They design the guardrail around the company's own risk, and never ask who outside the company absorbs the cost of a mistake.
They treat "the pilot went fine" as proof of safety, when a near miss caught by a hard rule looks identical, from the outside, to a problem that simply never had the chance to happen.
They widen the pilot based on overall accuracy instead of checking whether the one irreversible action ever slipped through, even once.

How to use it live. When someone asks you about a pilot with real-world side effects, ask yourself one thing out loud: which single action, if it went out by mistake, could not be quietly undone, and does the pilot block that one specifically, not just generally.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0