ConceptAdvancedAI Opportunity & Model Strategy / Build vs buy vs fine-tune decisions / #11
Describe the reversibility test you would apply to this decision.
ORDER a wire got stopped for six hours before anyone had asked how fast that stop could be undone
Quillhouse Bank screens outbound wire transfers for fraud. SentryWire is the model that scores each one. Soren Vasilenko is the AI PM who shipped it as a single, all-or-nothing gate. Colette Dumaine is the risk officer who took the phone call the day it blocked the wrong wire.
The direct answer
Before committing to any AI decision that acts on its own, ask three things: how fast could you detect if it made a bad call, how cheaply could you undo that call, and can it roll out on a small, reversible slice first instead of all at once. If the honest answers are "not fast," "not cheap," and "no, it's all-or-nothing," slow down, no matter how good the model's accuracy looks in a demo.
Do this, in order
Ship in shadow mode first: let the model score real decisions without acting on them.Why: it's the cheapest possible way to see how the model behaves against real outcomes, at zero reversal cost.
Move to advisory before automatic: show the score to a person who still makes the call.Why: a human in the loop is a second, cheap-to-reverse checkpoint before the model gets to act alone.
Automate only a small, low-stakes slice first, never the whole volume at once.Why: a bad call on 2 percent of traffic is a bug. A bad call on 100 percent of traffic is an incident.
Before full rollout, name exactly how you'd detect a bad call and how long that would take.Why: a decision nobody would notice going wrong for weeks is the one that needs the most staging, not the least.
Treat any step with a real contract, a wired-in pipeline dependency, or a customer-facing commitment as hardest to reverse, and sequence it last.Why: those are the specific things that turn a bad call into a bad quarter.
How to answer this, stage by stage
Nobody is scoring whether your model is accurate. They're scoring whether you'd know, and how fast, the day it isn't.
Stage 1
Scope it to one real decision
Say it like this
"I'll ground this in one case. Quillhouse Bank built SentryWire to score outbound wires for fraud risk. The question is how I'd test whether the way it's deployed, not just the model itself, can actually be walked back if it's wrong."
Why this works
Keeps "reversibility test" from becoming an abstract checklist with no real stakes behind it.
Stage 2
Say your structure out loud
Say it like this
"I'll run this as ORDER. Outcome, what the decision is actually competing to move. Reversibility, how cheap or expensive each rollout stage is to undo. Dependency, what has to be true before the next stage is even allowed. Evidence, what you can learn cheaply first. Rank, the actual sequence, and why."
Why this works
Signals a repeatable rollout method instead of a single yes-or-no judgment call.
Stage 3
Reframe: this isn't "is the model good enough"
Say it like this
"This isn't really about SentryWire's accuracy on a test set. It's about what happens the day it's wrong on a real wire, how fast anyone would find out, and how expensive it would be to take that call back."
Why this works
Moves the question from model quality, which is easy to answer with a metric, to deployment risk, which is the actual test being asked for.
Stage 4
Give the one decision: the test itself
Say it like this
"Three questions, every time. How fast would we detect a bad call? How cheap is it to undo? Can we roll it out on a small slice first? SentryWire failed all three on day one: it was wired directly into the approval pipeline, it blocked wires outright with no shadow period, and the first anyone heard of a bad call was an angry phone call, hours later."
Why this works
This is the direct answer, stated as a real, repeatable test rather than a one-off observation about SentryWire specifically.
Stage 5
Prove it with the compressed near miss
Say it like this
"SentryWire flagged a longtime corporate client's wire, tied to a real estate closing, as high risk and blocked it outright. It sat blocked for six hours before Colette even heard about it, because there was no advisory step, no human checkpoint, nothing between the score and the block."
Why this works
Turns "reversibility" from a theory into one specific, real wire that got stuck for six hours nobody could get back.
Stage 6
Name the AI-specific reasoning and the trade-off accepted
Say it like this
"The honest reason this matters more for a fraud model than most software is that its calls are probabilistic, not deterministic, it will sometimes flag a real client by mistake no matter how well it's tuned. We accepted a slower, staged rollout, shadow mode, then advisory, then a small slice, in exchange for never again finding out about a bad call from an angry client instead of a dashboard."
Why this works
Names the actual model-specific reason (probabilistic scoring, not a bug to patch) and states the trade-off, speed to launch against detection speed, out loud.
Stage 7
Say where the test can be lighter, then close
Say it like this
"A low-stakes internal tool, like a model that just labels which wires need extra documentation for recordkeeping, doesn't need this much staging, a wrong label costs a few minutes of rework, not a blocked wire. For anything that can act on money or a customer without a person in between, the full test applies. Same three questions, every time: how fast would we know, how cheap to undo, can we roll it out in a slice."
Why this works
Closes with real judgment about when to relax the test, and restates the direct answer in one breath.
Let's learn
What happens the moment a bank's fraud model gets to make a call it can't take back?
SentryWire scores every outbound wire transfer at Quillhouse Bank for fraud risk. Before it existed, a small compliance team manually reviewed roughly 800 wires a day, spending about 90 seconds per wire on a quick risk check, close to 20 hours of team time daily. SentryWire's launch version scored every wire instantly and, above a set risk threshold, blocked it outright, no human step in between.
SentryWire skipped straight to the last box. Nothing before it had ever been tested.
Weeks needed to reverse a bad call, by rollout stage
Cheap to reverseModerateExpensive
The reversal cost climbs sharply once contracts, integrations, and customer commitments are in place. That climb is exactly what the rollout order is meant to control.
Here's the turn: SentryWire's average accuracy was never really the problem, it caught real fraud attempts at a rate the team was happy with. The real problem was that nobody had asked how the deployment itself would behave the day it was wrong, and it was wired in a way that made a wrong call expensive and slow to catch.
We weren't testing whether the model could spot fraud. We were testing whether the bank could survive the day it spotted the wrong thing.
At its worst, this costs more than one blocked wire. A real client, mid-closing on a real estate deal, has their wire stopped with no explanation and no fast path to a human, and the story that reaches their relationship manager the next morning isn't "our fraud model caught something," it's "the bank froze our money for six hours for no reason."
The choice I would take back
Treating SentryWire's rollout as one unit of work, a single all-or-nothing switch flipped on for the entire wire volume at once, instead of a staged sequence with a shadow period and an advisory step first. That made sense when the team was racing to show fraud losses going down fast. It stopped making sense the moment a wrong call had no smaller, cheaper stage to have been caught in first.
What I would leave alone: a model that just labels which wires need extra documentation for recordkeeping doesn't need this level of staging. A wrong label costs a few minutes of rework, not a blocked wire and an angry client, so the full reversibility test would be overkill there.
The lesson: reversibility isn't a property of the model, it's a property of how you deploy it. The exact same SentryWire, run in shadow mode first, is a completely different risk than the one wired straight into the approval pipeline on day one.
Now here is the same thing as a story
The short version above is what you'd say out loud in the room. Read this one for what it actually felt like the day the wrong wire got stuck for six hours.
Soren Vasilenko had pushed hard to get SentryWire live before quarter's end, real fraud losses had been climbing, and the board wanted to see a number moving down. Colette Dumaine, the risk officer, had signed off on the launch plan, trusting the accuracy numbers Soren's team had shown her.
Nobody had asked which door SentryWire actually was before it shipped.
The launch went smoothly for three weeks. Then a wire from a longtime corporate client, tied to a same-day real estate closing, tripped SentryWire's threshold and got blocked automatically. There was no advisory step, no flag to a human, no fallback. The wire simply stopped moving.
We didn't build a model that made a mistake. We built a pipeline with no door back out of one.
Colette found out six hours later, when the client's furious relationship manager called her directly, not through any internal alert. By then, the closing had nearly fallen through, and Colette had no way to explain, in the moment, why the wire had been stopped or how long it would take to release it, because nobody had built a fast path to answer either question.
SentryWire's launch skipped straight to the bottom right corner of this chart, the single worst place to start.
Share of wire volume covered by auto-blocking, over the staged rollout
Share of volume on auto-block
Zero automated coverage for the first five weeks was the point, not a delay. Every percentage point after that was earned against real evidence.
Knowledge spark: what's the difference between shadow mode and advisory mode?
Shadow mode means the model scores real decisions but never acts, and nobody sees the score yet, it's pure measurement. Advisory mode means a person sees the score and still makes the final call. Only after both stages prove the model trustworthy does automatic action, without a person in the loop, become a reasonable next step.
Racing to ship SentryWire as one big launch had made sense the week fraud losses were climbing and the board wanted visible progress fast. It stopped making sense the moment a real, legitimate wire got caught with no cheap way to catch the mistake before it reached a client.
Four real questions, not a gut feeling about how mature the model seemed.
Here's the replay: Soren rebuilt the rollout as a real sequence. Two weeks in shadow mode first, comparing SentryWire's calls against real fraud outcomes with nothing blocked yet. Three weeks in advisory mode, where a human reviewer saw the score and made the actual call. Then automatic blocking, but only on wires under a set dollar threshold, the smallest, least damaging slice, for a full month before expanding further.
SentryWire's original launch hit three of these four signs at once. Nobody had checked for any of them.
Six months later, a similar borderline case came up, a large legitimate wire that scored just above the automatic-block threshold. Because it landed during the advisory-only phase, a reviewer caught it and cleared it in nine minutes. No client ever knew there had been a close call at all.
What I'd tell myself, hearing that furious relationship manager's call land on Colette's desk: fraud losses climbing was real pressure, and it was right to want to move fast. The mistake was treating "fast" and "all at once" as the same thing, when a staged rollout would have gotten there just as quickly, with a door back out at every step.
ORDER, run on SentryWire's own rolloutOne line per letter, if you want to say it fast under pressure.
O
Outcome. What is this decision actually competing to move?
Fewer real fraud losses, without wrongly stopping legitimate wires. Not "having an AI fraud tool live," a real, measurable bar on both sides of the error.
Without naming both sides of the outcome, a rollout plan is just a launch date with no real target.
R
Reversibility. Which rollout stage is cheapest to undo?
Shadow mode costs nothing to reverse, just stop logging. Advisory costs almost nothing, a person still decides. Auto-blocking wired into the live pipeline is the most expensive and slowest to catch, because by the time anyone notices, a real wire is already stopped.
This is the actual reasoning the whole test runs on, and the answer to the question being asked.
D
Dependency. What has to be true before the next stage is allowed?
Auto-blocking any slice only becomes allowed once shadow mode has proven precision against real outcomes, and advisory mode has run long enough to show the score is one a human would actually agree with most of the time.
Skipping a dependency is exactly how SentryWire ended up auto-blocking with zero evidence behind it.
E
Evidence. What could you learn cheaply before committing?
Two weeks of shadow-mode data, comparing SentryWire's calls against real, confirmed fraud outcomes, all before a single real wire was ever blocked.
Cheap evidence gathered before the expensive step is what turns a launch date into an actual decision.
R
Rank. State the sequence, and defend the first step.
Shadow mode, then advisory, then auto-block on a small dollar-threshold slice, then full rollout. Each step only unlocks once the one before it has proven itself against real outcomes.
This is the direct answer, run as an actual sequence, not a diagram nobody followed.
The recap, one line per letter: outcome is fewer real losses without wrongly blocking real wires, reversibility is why shadow mode goes first and full auto-block goes last, dependency is the proof each stage needs before the next one is allowed, evidence is the two weeks of shadow data gathered before anything real got blocked, and rank is the actual sequence that shipped: shadow, advisory, staged block, full rollout.
And if you want to be sure it really works, try it somewhere elseSame five letters, a ticketing company's pricing model instead of a bank's fraud model.
Priya Osei is the AI PM at Turnstile Events, a platform that helps venues price and sell event tickets. A team proposes letting a model set final ticket prices automatically based on real-time demand. Mapped onto ORDER: outcome is maximizing revenue without pricing real fans out or triggering refund disputes over sudden price jumps. Reversibility ranks the model suggesting a price to a human pricing manager as cheap and fast to undo, versus the model setting a live, customer-facing price directly, which is the hardest to reverse, since a ticket already sold at a bad price can't be quietly repriced afterward. Dependency: the model only gets to set a price directly once shadow-priced events, tracked internally with no price shown to buyers, show it agrees with human pricing decisions closely enough. Evidence comes from those shadow-priced events, run for a full ticketing cycle before any real price changes based on the model alone. Rank ends with the model suggesting a price for most of the catalog, and setting price directly only for one small, low-stakes event category, after six clean months of shadow and advisory data.
Same test, a different industry entirely: the hardest-to-reverse step, a customer-facing price, gets sequenced dead last here too.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "how fast would we detect a bad call, how cheap to undo, can it roll out in a slice, in that order, before committing to anything," and stop.
Cost: no time to run a real shadow period before a decision is due. Say so honestly, and make the shortest possible shadow window, even one week, the very next step, not a skip straight to launch.
The model got better, for real: SentryWire's accuracy improves substantially after a vendor update. Rerun the same staged rollout anyway, a better model still deserves the same test, since the risk being managed is the deployment, not the model's quality.
Where people run it wrong.
They test whether the model is accurate enough, and treat that as the same question as whether the rollout is safe to reverse.
They skip shadow mode because it "delays the launch," without pricing what a real, uncaught bad call costs instead.
They automate the whole volume at once instead of the smallest slice that would still prove the model out.
How to use it live. The moment someone proposes letting an AI decision act automatically, ask: "If it's wrong tomorrow, how would we know, and how fast could we undo it?" If nobody in the room can answer both halves, that's the actual next step, not the launch.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits describing a reversibility test for an AI decision?
Tap to flip
ANSWER
ORDER: outcome, reversibility, dependency, evidence, rank. It sequences a rollout by what's cheapest to undo, not by launch pressure.
2 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Soren Vasilenko, the AI PM who shipped SentryWire as a single all-or-nothing gate. Colette Dumaine is the risk officer who took the client's furious call six hours later.
3 · THE THREE QUESTIONS
What are the three questions the reversibility test actually asks?
Tap to flip
ANSWER
How fast would we detect a bad call? How cheap is it to undo? Can it roll out on a small slice first, instead of all at once?
4 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Treating SentryWire's rollout as one all-or-nothing switch for the whole wire volume, instead of a staged sequence with a shadow period and an advisory step first.
5 · THE NUMBER
Fill in the blank: the legitimate wire stayed blocked for ___ before Colette even heard about it.
Tap to flip
ANSWER
Six hours. There was no advisory step and no fast internal alert, so the first notice came from the client's furious relationship manager, not the system.
6 · THE REPLAY
A similar borderline wire comes up after the rollout got staged. What changes?
Tap to flip
ANSWER
It lands during the advisory-only phase, and a reviewer clears it in nine minutes. No client ever knows there was a close call at all.
7 · WHERE IT WOULDN'T MATTER
Name a decision at Quillhouse where this full test would be overkill.
Tap to flip
ANSWER
A model that labels which wires need extra recordkeeping documentation. A wrong label costs a few minutes of rework, not a blocked wire and an angry client.
8 · CROSS PRODUCT TRANSFER
Section 4 runs ORDER again on a different product. Which one, and what's the hardest step to reverse there?
Tap to flip
ANSWER
Turnstile Events' ticket pricing model. The hardest step to reverse is the model setting a live, customer-facing price directly, since a sold ticket can't be quietly repriced afterward.
Check yourself Score: 0 / 0
Short answer, name the test
1. Name the three questions the reversibility test asks before any AI decision is allowed to act on its own.
Show hint
Look at Stage 4 in the walkthrough, and the direct answer at the top.
Show answer
Model answer: How fast would we detect a bad call, how cheap is it to undo, and can it roll out on a small slice first instead of all at once.
Multiple choice
2. Why was SentryWire's launch version specifically hard to reverse?
A. It used a more expensive model than the team could afford long-term.
B. It was wired directly into the approval pipeline and blocked wires outright, with no shadow period or advisory step to catch a bad call first.
C. Regulators required a six-month notice period before any change to the model.
D. It only ran on outdated fraud data.
Show hint
Look at the quadrant diagram and where SentryWire's launch version sat on it.
Show answer
B. No shadow mode, no advisory step, and a direct wire into the live pipeline meant a bad call became a real, stuck wire before anyone could check it.
True or false
3. True or false: SentryWire's core problem was that its fraud-catching accuracy was too low.
True
False
Show hint
Look at the block-highlight in Let's learn.
Show answer
False. Its average accuracy was fine. The real problem was that nobody had tested how the deployment itself would behave the day it made a wrong call.
Fill in the blank
4. Fill in the blank: after the rollout was restaged, a similar borderline wire was caught and cleared by a human reviewer in just ___ minutes.
Show hint
Look at "here's the replay" near the end of the story.
Show answer
Nine minutes. It landed during the advisory-only phase, so a person, not the model alone, made the final call.
Short answer, apply it yourself
5. Think of an automated decision you've encountered, a flight rebooking, a loan denial, an account lock. If it had been wrong, how would you have found out, and how long would it have taken to undo?
Show hint
Think about whether there was a clear, fast path to a human, or whether you had to escalate blindly.
Show answer
Model answer: An account lock with no visible reason and only a generic support queue to escalate through could easily take days to undo, exactly the kind of decision a staged, reversible rollout is meant to prevent.
Short answer, work the number
6. If SentryWire's shadow-mode period had run for only two days instead of two weeks, would the team have had enough evidence to trust the auto-block rollout?
Show hint
Think about how many real wires, and how many real fraud cases, likely occur in just two days versus two weeks.
Show answer
Model answer: Probably not. Two days likely wouldn't surface enough real fraud cases or borderline calls to know how the model behaves on the cases that actually matter, so the evidence step would still be too thin to trust.
Before you close the answer
Why this works
Tests whether you can separate model quality from deployment risk. Most candidates answer "is the model good enough" when the real question is "what happens when it isn't."
Follow-up traps
"Doesn't shadow mode just delay catching real fraud?" Response: only briefly, and the cost of a few extra days of manual review is far smaller than the cost of a real client's wire getting stuck for six hours with no fast path back.
"What if the business needs it live immediately for a regulatory deadline?" Response: even a one-week shadow period beats none, and advisory mode can launch fast while still keeping a human as the real decision-maker until the model earns full trust.
If pressed
The staged rollout also added a same-day manual override path, so even after auto-blocking launched on the small slice, any flagged wire could be released by a risk officer within thirty minutes, not six hours, if it turned out to be a real client after all.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
