CaseAdvancedAI Opportunity & Model Strategy / Roadmapping under model uncertainty / #14
How do you protect a roadmap from being rewritten every time a new model launches?
FLIPSthe 48 hours nobody built between a headline and a rewrite
Larkspur Residential runs FixQueue, a tool that reads a tenant's photo and two sentences about a leaking faucet or a broken thermostat, then ranks the ticket and routes it to the right vendor. Ijeoma Adeyemi built its roadmap. Every time a rival ships something flashier with a new model, her CEO takes the roadmap back and rewrites it himself, alone, over a weekend.
The direct answer
Don't protect the roadmap by resisting change. Build a 48-hour triage memo: a fixed, one-page test of whether the new model actually touches either of the two bets the roadmap is built on, run against your own held-out ticket data, not the vendor's benchmark. Give someone below the CEO standing authority to close anything under a set size without reopening the document. Most launches turn out to change nothing that matters; the memo is what proves it fast enough that nobody needs to rewrite anything to feel safe.
Do this, in order
Build the 48-hour triage memo and give someone standing authority to close small changes with it.Why: without a fast, trustworthy answer, the only two options left are ignore it or rewrite everything, and panic always beats ignoring.
Test every new model against your own eval set, never the vendor's numbers alone.Why: a benchmark can be true in general and false on your actual mix of tickets.
Track hours spent re-litigating the roadmap, by quarter, as a standing number.Why: it turns "the roadmap keeps changing" from a feeling into something you can catch climbing before it costs a whole weekend.
Keep a short log of what each past launch actually changed.Why: after three or four entries that say "changed nothing," the pattern itself becomes the calming argument.
Leave routine vendor patch notes outside the memo entirely.Why: not every update is a frontier launch, and running the full gate on all of them just teaches people to skip it.
How to answer this, stage by stage
Nobody is scoring whether you can define a roadmap. They're scoring whether you can name the 48 hours that were missing.
Stage 1
Scope it to one product and one person
Say it like this
"I'll ground this in FixQueue at Larkspur Residential, and the actual quarter their CEO started rewriting the roadmap himself every time a rival's model news hit the trade press."
Why this works
Keeps the answer from turning into a general essay on "resisting change."
Stage 2
Say the structure out loud
Say it like this
"I'll run this as FLIPS. Find the person, locate the habit that was working, identify the exact behavior that snaps, pinpoint the old decision behind it, then show the replay with a fix in place."
Why this works
Two seconds that tell the interviewer you have a method, not just an opinion.
Stage 3
Reframe: this isn't about stopping change, it's about the missing 48 hours
Say it like this
"The real question isn't 'how do I stop the CEO from reacting to new models.' It's 'what fast, trustworthy answer can he get in 48 hours instead of a weekend of rewriting.' Without that answer, reacting is the only option he has."
Why this works
This is where a strong answer separates from "communicate the roadmap better."
Stage 4
Give the one decision
Say it like this
"Build a one-page triage memo, run within 48 hours of any model launch, tested against Larkspur's own held-out tickets, not the vendor's benchmark. Give someone below the CEO the authority to close anything under about two engineer-weeks without reopening the roadmap."
Why this works
This is the direct answer, stated as a specific, buildable thing instead of a value about patience.
Stage 5
Prove it with the compressed failure
Say it like this
"When Birchgate, a rival, got a trade-press write-up for an instant repair-diagnosis feature, Roland saw it on a Sunday and spent the weekend rewriting Larkspur's whole roadmap himself. Ijeoma's team spent the next six weeks re-scoping work that had already been agreed to."
Why this works
Compresses the whole failure into the exact gap a 48-hour memo would have closed.
Stage 6
Name the AI-specific trap inside the fix itself
Say it like this
"The memo can't just read the vendor's benchmark and say 'looks better.' A model that beats last year's model on a general leaderboard can still be worse on Larkspur's actual mix of plumbing photos and thermostat complaints, so the memo has to run the new model against Larkspur's own held-out tickets before anyone trusts the number."
Why this works
Shows the guardrail is specific to how models get evaluated, not a generic process step.
Stage 7
Close on the one line
Say it like this
"So the fix isn't a calmer CEO. It's a 48-hour answer he can trust instead of a weekend he has to spend rewriting things himself to feel safe."
Why this works
Restates the direct answer in one breath and ends the answer on the decision, not the story.
Let's learn
Here is what happens when a plan survives two years of quiet trust and then gets rewritten in a single weekend by the one person who agreed to stop touching it.
Before FixQueue, a Larkspur maintenance coordinator read every tenant message by hand, guessed at urgency, and called around for an available vendor, about twenty-five minutes a ticket. FixQueue reads the photo and the message, ranks the ticket, and suggests a vendor in under a minute. Ijeoma shipped it behind a roadmap with a fixed cadence: a quarter of planned work, a small change-budget for anything urgent, and a monthly check-in instead of daily requests from the exec team.
Five questions, in order. Only one of them, I, is genuinely hard to answer.
For two years, that cadence worked. Roland Kessler, Larkspur's CEO, stopped emailing Ijeoma at 11pm with "have you seen this new model" messages. He let quarters run. He asked questions in the monthly check-in instead of mid-sprint.
Here's the turn: the habit that broke wasn't Roland losing interest in the roadmap. It was the opposite: he trusted it so completely that he had no small, fast way to react when something felt urgent. So when a rival's launch finally spooked him, there was no dial between "let it run" and "take it back and redo the whole thing myself." He only had the second option, because nobody had ever built the first.
Hours spent rewriting the FixQueue roadmap, by quarter
Two calm years, then one weekend that cost more than the previous seven quarters combined. The memo brought it back down without banning anyone from reacting.
At its worst, this doesn't just cost a weekend. Six weeks after the rewrite, half of the quarter's original work had to be re-scoped, re-estimated, and re-explained to engineers who had already planned their sprints around the version that existed before Roland touched it.
Share of roadmap items shipped as originally scoped, trailing six quarters
The hours chart shows the cost of the weekend itself. This one shows the cost that followed it: a full quarter where nearly half the roadmap stopped shipping as planned, recovering only after the memo gave Roland a faster answer than a rewrite.
The choice I would take back
Larkspur never built a step between "a new model exists" and "the roadmap changes." There were only two paths: ignore the news, or open the whole document. That made sense when model launches were rare and mostly irrelevant to tenant photo classification. It stopped making sense the moment launches became monthly and the CEO started reading trade press on Sundays.
What I would leave alone: routine vendor patch notes, the small monthly updates that don't touch the two bets the roadmap depends on, don't need the 48-hour memo at all. Running the full process on every minor release just teaches people to skip it when it actually matters.
The lesson: a roadmap doesn't get rewritten because someone stops caring about it. It gets rewritten because the only fast option left is the biggest one. Build a small one, and the big one stops being anyone's only move.
Now here is the same thing as a story
The short version above is what you'd say defending a roadmap decision to your own exec team. Read this one for how a quiet Sunday habit turned into a six-week mess.
Sunday nights at Larkspur used to be quiet for Ijeoma. Roland read industry newsletters on the couch, and for two years, whatever he read there stayed there until the Monday check-in, if it came up at all.
Ijeoma had built that habit on purpose. FixQueue's roadmap ran on a fixed quarterly cadence: a locked set of bets, a small change-budget for anything genuinely urgent, and a monthly review instead of a running conversation. Early on, Roland pushed back on almost everything. By the second year, he'd stopped. The plan kept working. Tenants got routed faster. Vendors showed up for the right jobs. He let it run.
For two years, Roland's trust moved like the left panel. It broke like the right one.
Then came a Sunday like the others, except for one headline. Birchgate Property Group, a rival two towns over, announced an "instant repair diagnosis" feature built on a brand-new multimodal model, and a trade outlet ran a glowing writeup with a screenshot.
Knowledge spark: why can't a benchmark answer this by itself?
A new model's score on a public leaderboard measures how it does on a broad mix of test questions nobody at Larkspur wrote. It says nothing about how it does on a leaking-faucet photo taken sideways in bad light, which is most of what FixQueue actually sees. Only a test against your own real tickets tells you if the new model matters here.
Roland didn't wait for Monday. He spent the weekend rewriting the roadmap himself, adding a rushed "instant diagnosis" initiative, reshuffling three planned features to make room, and handing the new document to Ijeoma's team on Monday morning as final.
Two years of the plan holding, undone in the same weekend as the headline that triggered it.
Nobody at Larkspur decided, on any single day, that Roland should get to redo two years of planning discipline over one article. But structurally, he only ever had two settings: let the roadmap run untouched, or take it back completely. There was no third option sized for "check if this actually matters," so the moment he felt he had to act, the only action available was the big one.
We didn't lose a plan that weekend. We lost the six weeks after it, re-arguing decisions the team thought were already made.
Engineers who had scoped their sprints around the old roadmap spent the next month and a half re-estimating work under the new one, much of which quietly reverted back to the original plan once someone finally tested the new model against Larkspur's own tickets and found it barely moved routing accuracy at all.
Ijeoma designed for a dial. What Larkspur actually had was a switch with no setting in between.
So here is the decision I would take back: build the 48-hour triage memo before the next headline, not after the next rewrite.
The gate that was missing sits exactly where the whole weekend happened instead.
With the memo in place, the replay runs differently. Birchgate's launch hits on a Sunday. Roland forwards the article Monday morning instead of opening the roadmap document himself. The memo comes back by Wednesday: the new model helps with photo classification cost, a little, but doesn't move either of FixQueue's two real bets, routing accuracy and vendor trust, on Larkspur's own tickets. The roadmap changes by exactly one line, a vendor swap for a small cost saving. Everyone else's sprint holds. And the thing I'd tell myself, if I could go back: Roland was never the problem. The problem was that I never gave him anything faster than "trust me" or "let me take it all back."
The five steps, if you want to remember it
F
Find the person. Whose Sunday is this?
Roland Kessler, CEO of Larkspur Residential, reading trade newsletters on the couch, two years into trusting a roadmap he stopped touching.
Naming him, not "leadership," is what makes the flip concrete instead of a complaint about executives in general.
L
Locate the habit. What did he stop doing because it worked?
He stopped pushing back on the quarterly plan mid-quarter. The plan kept delivering, so he let go of daily involvement.
That habit was the actual product Ijeoma had shipped: a CEO who didn't need to hover.
I
Identify the flip. What verb snaps?
He goes from letting the roadmap run untouched to taking it back and rewriting it himself, alone, in one weekend. No middle setting existed.
This is the hard step, and the one the whole answer turns on.
P
Pinpoint the old decision. Which choice only made sense before?
Larkspur never built a fast, small step between "a new model exists" and "the roadmap changes." That gap was invisible when launches were rare.
A specific, reversible product decision, not a vague call to "communicate better."
S
Show the replay. Same headline, new design.
With the 48-hour memo, the roadmap changes by one line by Wednesday instead of being rewritten wholesale over a weekend.
A countable result: one line changed, not the whole document, and no sprint gets re-scoped.
The recap, one line per letter: find is Roland on a Sunday couch, locate is the habit of letting a quarter run untouched, identify is the switch between full trust and a full weekend rewrite, pinpoint is the missing 48-hour gate, and show is the same headline landing as one changed line instead of a torn-up roadmap.
And if you want to be sure it really works, try it somewhere else
Fernhollow Veterinary Partners runs a scheduling assistant that reads a pet owner's message and suggests an appointment slot and urgency level. Tobias Renwick, the clinic's scheduling coordinator, used to log every time he overrode the assistant's suggestion in a shared notes file the whole front desk could read, so the pattern of overrides was visible to everyone. Mapped onto FLIPS, the person is Tobias, the habit is logging overrides openly, and the flip here is a different family entirely: a concealment flip. After the clinic's owner personally rewrote the whole scheduling roadmap the first time a scheduling vendor announced a flashier new model tier, unprompted, Tobias quietly stopped logging his overrides, worried that a visible pattern of "the assistant got it wrong again" would trigger another unplanned rewrite. Pinpoint lands on the same root cause as Ijeoma's story: no small, fast, pre-agreed gate existed between a vendor's announcement and a full re-plan, so the owner's only move, once spooked, was to take the whole thing back himself. Show the replay: with a triage step in place, Tobias goes back to logging every override in the open, because a vendor's announcement now gets a calm 48-hour answer instead of an unplanned rewrite that makes visible mistakes feel dangerous to report.
Three of the four branches are the design working. The fourth branch is the same missing gate, in a different building.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "build a 48-hour triage memo tested on your own data, with someone below the CEO able to close small changes," and stop.
Cost: no budget for a formal process this quarter. Say so honestly, and start with just the one-page memo template and a named owner, since that costs almost nothing and stops the biggest source of wasted hours.
The model gets better, for real: if a new model genuinely does move one of the two real bets, that's the memo working as intended, and the honest move is to open the roadmap on purpose, not because someone panicked over a headline.
Where people run it wrong.
They try to ban the CEO from reacting to industry news, which never survives contact with a real headline.
They test the new model against the vendor's own benchmark instead of their own tickets, and greenlight a change that never actually helps.
They wait for the next rewrite to build the memo, instead of building it before the next headline lands.
How to use it live. The moment someone asks about protecting a roadmap from constant rewrites, ask yourself: what's the fastest trustworthy answer this person could get instead of taking the whole thing back themselves? Build that, and the rest of the answer follows.
Flashcards (tap any card to flip it)
1 · THE FLIP FAMILY
What flip family is this?
Tap to flip
ANSWER
Delegation flip: Roland let Ijeoma own the roadmap for two years, then took it back and rewrote it himself the moment a rival's launch spooked him.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Roland Kessler, CEO of Larkspur Residential, who spent two years letting a quarterly roadmap run without touching it.
3 · THE HABIT
What did Roland stop doing during the two calm years?
Tap to flip
ANSWER
He stopped pushing back on the roadmap mid-quarter and let the fixed cadence run, asking questions only at the monthly check-in.
4 · THE FLIP, IN THIS STORY
What's the two-setting switch here?
Tap to flip
ANSWER
Letting the roadmap run completely untouched, versus taking it back and rewriting it entirely himself. No setting in between existed.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Never building a small, fast step between "a new model launched" and "the roadmap changes," which made sense when launches were rare and irrelevant.
6 · THE NUMBER
Fill in the blank: rewriting the roadmap after Birchgate's launch cost Larkspur ___ hours in that quarter, versus 3 to 4 hours in a normal quarter.
Tap to flip
ANSWER
46 hours, which dropped to 6 hours the quarter after the 48-hour triage memo was introduced.
7 · THE REPLAY
Same headline, new design, what changes?
Tap to flip
ANSWER
Roland forwards the article instead of opening the roadmap. The triage memo comes back Wednesday: the new model doesn't move either real bet. One line changes. Every other sprint holds.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this same question again for a different product, with a different flip family. Which product, and which family?
Tap to flip
ANSWER
Fernhollow Veterinary Partners' scheduling assistant. The flip is concealment: Tobias Renwick stopped logging his overrides once an unplanned rewrite made visible mistakes feel dangerous to report.
Check yourself Score: 0 / 0
Multiple choice
1. Why did Roland only have two options, letting the roadmap run or rewriting all of it, instead of something in between?
A. Because Ijeoma refused to make any changes to the roadmap once it shipped.
B. Because Larkspur never built a small, fast step for checking whether a new model actually mattered before anyone touched the roadmap.
C. Because the roadmap software only allowed full rewrites, not partial edits.
D. Because Roland didn't trust Ijeoma's team to make any changes at all.
Show hint
Look at the "pinpoint" step and "the choice I would take back."
Show answer
B. With no small gate in between, the only two moves left were ignoring the news or opening the whole document.
True or false
2. True or false: this answer argues the fix is convincing Roland to stop reading industry news about new models.
True
False
Show hint
Look at the direct answer and "where people run it wrong."
Show answer
False. The fix is giving him a fast, trustworthy 48-hour answer instead of banning the reaction that's coming regardless.
Fill in the blank
3. Fill in the blank: the triage memo has to test a new model against Larkspur's own ___, not the vendor's published benchmark.
Show hint
Look at the knowledge spark and stage 6 of the walkthrough.
Show answer
Held-out tickets. A general leaderboard score says nothing about a sideways photo of a leaking faucet in bad light.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when it was first made?
Show hint
Look at "the choice I would take back."
Show answer
Model answer: Never building a fast step between a model launch and a roadmap change. It made sense when launches were rare enough that skipping the question entirely cost nothing.
Short answer, apply it yourself
5. Pick a plan you've made yourself, at work or otherwise. What's one habit of trusting it that would break if something changed enough, and what small check could you build now instead of redoing the whole plan later?
Show hint
Think of a budget, a training schedule, or a project timeline you stopped double-checking once it was working.
Show answer
Model answer: A monthly budget you stopped reviewing line by line once it worked for a year; a quick check against one new large expense beats redoing the whole budget from scratch.
Short answer, where it wouldn't matter
6. Name a case at Larkspur where a new model launch genuinely wouldn't need the 48-hour memo at all.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: A routine vendor patch note that doesn't touch routing accuracy or vendor trust. Running the full memo on every minor release just trains people to skip it.
Before you close the answer
Why this works
Tests whether you'll treat "protect the roadmap" as resisting a CEO's instinct, or as building the one fast answer that makes the instinct unnecessary.
Follow-up traps
"What if the CEO overrides the memo anyway and rewrites it himself regardless?" Response: then the real gap is authority, not process; the memo needs a pre-agreed rule that anything under its threshold doesn't get reopened, decided before the next headline, not during it.
"Isn't 48 hours still too slow if a real competitive threat exists?" Response: the memo doesn't block a real threat, it filters out the many launches that aren't one; a genuine threat still moves fast, it just moves through the memo instead of around it.
If pressed
The actual memo Larkspur settled on scores three things against held-out tickets: routing accuracy shift, cost per classification, and vendor-match agreement rate, and only a move of more than two points on any of the first two triggers a full roadmap conversation.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.