InterviewAdvancedAI Opportunity & Model Strategy / Roadmapping under model uncertainty / #23
Present a six-month AI roadmap for a product I name and defend the sequencing.
ORDERthe six months that ranked a confidence flag above every feature leadership actually wanted
The interviewer names the product: ClimaSense, an AI tool at Brindlemere Home Services that reads a technician's photo and notes on a broken heating or cooling unit and suggests a likely diagnosis and the part to bring back next time. Idris Wechsler is the AI PM asked, on the spot, to present the next six months and defend why anything comes before anything else.
The direct answer
Sequence by which item is hardest to undo if skipped, not by what leadership is most excited about. Ship the confidence flag before any new equipment type, prove electric AC on a small crew before gas furnaces, and hold gas furnace diagnostics to its own safety bar before it ever reaches a technician's phone. Parts prediction and fleet-wide rollout come last, because they only make sense once the diagnosis underneath them can actually be trusted.
Do this, in order
Build the confidence flag before touching a second equipment type.Why: without it, every future expansion inherits the same risk of a wrong diagnosis reaching someone's hands with no warning attached.
Prove electric AC on a small crew before claiming it for the fleet.Why: one technician's good week could be luck. A crew's month is a pattern worth building on.
Hold gas furnace diagnostics to its own safety bar, separate from electric AC's.Why: a furnace miss can mean a cracked heat exchanger and a carbon monoxide risk, not just a wasted truck roll.
Add parts prediction only once diagnosis is trusted on that equipment type.Why: predicting the wrong part before the diagnosis is solid just compounds one bad guess with another.
Roll out fleet-wide last, with the callback rate as a standing alarm, not a launch metric.Why: scale is the one move that can't be undone quietly if something's still wrong underneath it.
How to answer this, stage by stage
Nobody is scoring whether your six months has six neat boxes. They're scoring whether you can defend which box comes first under real pressure.
Stage 1
Scope it to the product you were just handed
Say it like this
"I'll build this for ClimaSense at Brindlemere Home Services, an AI diagnosis assistant for HVAC technicians, since that's the product on the table."
Why this works
Shows you can commit to the specific product handed to you instead of retreating to something generic.
Stage 2
Say your structure out loud
Say it like this
"I'll rank this with ORDER. Outcome, what all six months are actually trying to move. Reversibility, which item is hardest to undo. Dependency, what unblocks what. Evidence, what I'd learn cheap first. Rank, the actual sequence, defended."
Why this works
Tells the interviewer you have a repeatable way to sequence, not a list ranked by gut feel.
Stage 3
Name the outcome, plainly
Say it like this
"Everything in these six months is competing to raise first-visit fix rate and cut diagnostic time, without a wrong diagnosis ever reaching a home unverified."
Why this works
Without naming this, six months of features is just a list, not a plan.
Stage 4
Give the sequence, defended by reversibility
Say it like this
"Months one and two: electric AC only, with a confidence flag built in from day one. Month three: pilot on one crew. Month four: gas furnace gets its own safety bar before it ships. Months five and six: parts prediction, then fleet-wide, with callback rate tracked the whole way."
Why this works
This is the direct answer, said as an actual sequence instead of a list of features with no order to them.
Stage 5
Prove it with the compressed failure
Say it like this
"When an earlier version expanded to gas furnaces on the same calendar as electric AC, ClimaSense once called a cracked heat exchanger an igniter failure. The technician fixed the wrong part. The customer's carbon monoxide detector went off a week later."
Why this works
Grounds the sequencing decision in a real, specific failure instead of an abstract "safety matters" line.
Stage 6
Name the trade-off you're accepting
Say it like this
"This means gas furnace coverage, which leadership wants badly before winter, slips by six to eight weeks. I'd rather miss that window than ship a diagnosis tool for a safety-critical system with no bar behind it."
Why this works
Says plainly what the sequencing costs, instead of pretending the safer order is free.
Stage 7
Say what you'd measure past launch
Say it like this
"I'd track callback rate by equipment type every week, not just at launch, since that's the number that would have caught the furnace problem before a detector did."
Why this works
Shows the plan doesn't end at rollout, it keeps watching the exact thing that broke last time.
Stage 8
Close on the one line
Say it like this
"Sequence by what's hardest to undo, not what's most requested. A confidence flag and a per-equipment safety bar come before any new feature, because those are the two things a fast expansion can't buy back afterward."
Why this works
Restates the direct answer in one breath, exactly what a live defense of the sequencing needs to end on.
Let's learn
Here is what happens when a roadmap's sequencing gets set by a launch date instead of by what each step actually depends on.
Before ClimaSense, a Brindlemere technician diagnosed a broken unit from experience alone: the sound a compressor makes right before it fails, the smell of a cracked heat exchanger, a hundred small signals nobody wrote down. First-visit fix rate across the fleet sat at 68 percent, meaning almost one in three calls needed a second truck roll. With ClimaSense suggesting a diagnosis from a photo and a few notes, that number started to move.
The roadmap's first version only ever shipped the left one.
Here's the turn: on electric AC units, where Brindlemere had years of clean repair data, ClimaSense's suggestions were genuinely strong. Leadership wanted that same win before winter, on gas furnaces, where a wrong call isn't just an extra truck roll, it's a missed crack in a heat exchanger that leaks carbon monoxide into someone's house.
First-visit fix rate, electric AC calls, month by month after ClimaSense launched
This was the real, honestly earned result. It only ever described electric AC, never gas furnaces.
At its worst, sequencing by a calendar date instead of a dependency doesn't just risk a bad launch. On a safety-critical system, it risks a family breathing carbon monoxide because a roadmap treated two very different kinds of equipment as one line item.
The choice I would take back
The early version of ClimaSense gave one flat diagnosis with no way for a technician to flag which part of it was wrong, so any correction meant redoing the whole call note from scratch. That made sense when the tool only handled electric AC, where a miss was rarely more than an extra part on the truck. It stopped making sense the moment gas furnaces joined the same tool with the same all-or-nothing correction.
What I would leave alone: I wouldn't slow down electric AC's rollout to wait for gas furnace's safety bar. That equipment type had already earned its pace, and holding it back for a problem it didn't have would cost trust for no reason.
The lesson: a roadmap's order isn't a scheduling exercise. It's a statement about which mistake you're willing to risk making in public, and which one you're not.
Now here is the same thing as a story
The short version above is what you'd say defending the sequence to leadership. Read this one for how trust broke somewhere it hadn't even been tested.
For eight months, the best part of Rosalind Ferber's day was pulling her phone out mid-repair and seeing ClimaSense's suggestion match exactly what she already suspected. She'd worked electric AC calls for six years, and the tool had never once sent her chasing the wrong part.
Then Brindlemere expanded ClimaSense to gas furnaces, three weeks ahead of leadership's committed date, before its own safety bar had actually been cleared.
One of these costs a delay. The other one costs something that can't be taken back.
On a different crew, a technician followed ClimaSense's suggestion on a gas furnace call: igniter failure. It was actually a cracked heat exchanger. The igniter got replaced. The furnace kept running. A week later, the homeowner's carbon monoxide detector went off in the middle of the night.
Knowledge spark: why is a heat exchanger crack so much worse than most HVAC misses?
A cracked heat exchanger can let carbon monoxide, an odorless gas, leak into a home's air supply. Most HVAC misdiagnoses cost a wasted trip. This kind can cost someone's safety, which is why it needs its own, higher bar before a tool gets to suggest it at all.
Nobody on Rosalind's crew had touched that call. But word of it moved through the fleet by the end of the week, the way bad news about a shared tool always does. Rosalind started double-checking every ClimaSense suggestion by hand again, even on electric AC, the equipment type where the tool had never once been wrong.
The furnace never touched Rosalind's crew. The doubt did, because trust in a shared tool doesn't stay in the equipment category where it broke.
Fleet-wide callback rate, which had been sitting near 9 percent for months, climbed on gas furnace calls specifically to 24 percent over six weeks, as the same kind of miss repeated on a handful of other calls before the expansion got paused. Electric AC's own callback rate never moved. It didn't need to. The damage to trust had already crossed the line between equipment types anyway.
The item in the top right should have shipped before gas furnaces ever did.
ORDER, the sequence a shared tool's trust actually depends onNot a features list with dates attached. ORDER is what decides which mistake you can afford to risk making first.
O
Outcome. What all six months are competing to move.
First-visit fix rate and diagnostic time, without a wrong diagnosis reaching a home unverified.
Without this, a six-month plan is just a list with dates, not a defended sequence.
R
Reversibility. Which decision is hardest to undo.
Expanding to gas furnaces without a confidence flag is hardest to undo, since a miss there can reach a family's air supply, not just a technician's truck.
This is the hardest step, and the one this whole sequence turns on.
D
Dependency. What unblocks what.
Parts prediction only makes sense once diagnosis is trusted. Gas furnace diagnosis needs its own safety bar before it can ship at all.
Neither of those later items is safe to build on top of a foundation that hasn't been proven yet.
E
Evidence. What you'd learn cheaply first.
A small-crew pilot on electric AC, one month, tells you whether the confidence flag actually changes technician behavior before committing to a fleet-wide claim.
Cheap evidence beats a leadership deadline as the thing that decides pace.
R
Rank. The actual six-month sequence, defended.
Confidence flag and electric AC pilot first, gas furnace's own safety bar fourth, parts prediction and fleet-wide rollout last.
The order follows what each step depends on, not which feature leadership is most eager to announce.
The recap, one line per letter: outcome is fix rate and diagnostic time without an unverified miss reaching a home, reversibility is that a furnace miss with no confidence flag is the hardest thing to undo, dependency is that parts prediction and fleet-wide rollout both sit on top of a diagnosis that has to be trusted first, evidence is a small-crew pilot before any fleet-wide claim, and rank is the confidence flag and electric AC first, gas furnace's safety bar fourth, scale last.
And if you want to be sure it really works, try it somewhere elseSame five letters, a school district's attendance-risk model instead of an HVAC tool. A different old decision breaks the second rollout.
Kestervane School District runs a model that flags students at rising risk of chronic absence, so counselors can reach out earlier. Its roadmap sequenced a district-wide rollout across all grade levels on the same calendar as the initial middle-school pilot. Mapped onto ORDER: outcome is getting counselors to the right student sooner, without a false flag following a kid around their file. Reversibility is highest for high-school rollout, since a wrongly flagged teenager can carry that label into a permanent record far longer than a wrongly flagged nine-year-old's file gets revisited. Dependency is that any grade band's rollout depends on that band's own false-flag rate being checked, since attendance patterns look completely different in kindergarten than in eleventh grade. Evidence is a one-semester pilot per grade band before expansion. Rank puts high school last, not first, reversing what the original calendar had planned. The old decision here isn't a missing confidence flag, it's a default setting: the model's flag threshold was tuned once, on middle-school data, and quietly carried over into every other grade band without being re-checked, because retuning felt like unnecessary extra work when the middle-school numbers already looked strong.
The same four branches decide a furnace diagnosis and a school district's flag alike.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "the confidence flag and each equipment type's own safety bar ship before any new feature, because those are what can't be bought back once trust breaks," and stop.
Cost: leadership says there's no budget for a separate gas furnace eval this quarter. Say so honestly, and hold gas furnace at electric-AC-only scope until the budget exists, rather than shipping it unevaluated.
The model gets better, for real: if gas furnace diagnosis genuinely clears its bar early, that's the honest trigger to move up the timeline, not a reason to have skipped the bar in the first place.
None of these four existed before the furnace incident. All four exist in the sequence now.
Where people run it wrong.
They let a leadership-announced date decide the order instead of asking what each item actually depends on.
They treat trust in a shared tool as scoped to one equipment type, when a single bad miss travels across the whole fleet.
They copy a threshold or a bar from one context into a new one without re-checking whether it still holds.
How to use it live. The moment an interviewer hands you a product and asks for a sequence, ask yourself: which item, if it goes wrong, can't be quietly walked back. Put that one, and whatever it depends on, first.
Gas furnace's own bar sits fourth on purpose, not because it was less wanted, but because it depended on everything before it.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits "present a roadmap and defend the sequencing"?
Tap to flip
ANSWER
ORDER: outcome, reversibility, dependency, evidence, rank. It sequences by what's hardest to undo, not by what's most requested.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Rosalind Ferber, a six-year Brindlemere technician who trusted ClimaSense's electric AC suggestions completely, checking her phone mid-repair.
3 · THE HABIT
What did Rosalind start doing again after hearing about the furnace incident?
Tap to flip
ANSWER
She started double-checking every ClimaSense suggestion by hand again, even on electric AC calls, the equipment type where the tool had never actually been wrong.
4 · THE DEPENDENCY
Why does parts prediction have to wait until last?
Tap to flip
ANSWER
Because predicting the wrong part before the diagnosis underneath it is trusted just compounds one bad guess with another.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Giving one flat diagnosis with no way to flag which part was wrong, a call that made sense when only electric AC was involved and stopped making sense once gas furnaces joined the same tool.
6 · THE NUMBER
Fill in the blank: gas furnace callback rate climbed to ___ percent over six weeks after the unevaluated expansion.
Tap to flip
ANSWER
24 percent, up from a fleet-wide baseline near 9 percent, while electric AC's own callback rate never moved.
7 · THE REPLAY
Same leadership pressure to hit gas furnaces before winter, but the safety-bar sequencing is already in place. What changes?
Tap to flip
ANSWER
Gas furnace diagnostics ship six to eight weeks later, once its own bar clears. No heat exchanger gets misdiagnosed, no detector goes off, and Rosalind never has a reason to stop trusting electric AC.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this same question again for a different product. Which product, and what old decision gets taken back?
Tap to flip
ANSWER
Kestervane School District's attendance-risk flag. The reversal is a default setting: the flag threshold was tuned once on middle-school data and carried over into every other grade band unchecked.
Check yourself Score: 0 / 0
Multiple choice
1. Per this answer, why does the confidence flag get sequenced before any new equipment type?
A. Because it's the cheapest feature to build.
B. Because every future expansion inherits the same risk of an unverified wrong diagnosis reaching someone, unless the flag exists first.
C. Because leadership specifically requested it first.
D. Because technicians prefer using it over any other feature.
Show hint
Look at the reversibility step and the quadrant diagram.
Show answer
B. The confidence flag sits in the top-right of the quadrant precisely because everything after it depends on it existing.
True or false
2. True or false: this answer argues gas furnace diagnostics should never ship at all, given the risk involved.
True
False
Show hint
Look at the six-month sequence's month four.
Show answer
False. Gas furnace diagnostics ship in month four, once it clears its own separate safety bar, not never.
Fill in the blank
3. Fill in the blank: first-visit fix rate on electric AC calls rose from 68 percent before launch to ___ percent by month four.
Show hint
Look at the line chart in Section 1.
Show answer
81 percent. This was the real, honestly earned number, and it only ever described electric AC.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when it was first made?
Show hint
Look at "the choice I would take back."
Show answer
Model answer: Giving one flat diagnosis with no way to flag which part was wrong. It made sense when only electric AC was involved, where a miss rarely cost more than an extra part on the truck.
Short answer, apply it yourself
5. Think of a tool you trust in one specific situation. If you heard it badly failed someone else in a completely different situation, would your trust in your own use of it change? Why or why not?
Show hint
Think about a GPS app, a spell-checker, or a recommendation engine that failed a friend on a task you've never actually asked it to do.
Show answer
Model answer: Hearing a GPS app sent a friend down a closed road might make you double-check its next few suggestions, even on a route it's always gotten right before.
Short answer, where it wouldn't matter
6. Name a decision in this roadmap where waiting for gas furnace's safety bar genuinely doesn't need to slow anything down.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: Electric AC's own rollout pace. It already earned its speed on its own evidence, and holding it back for a furnace-specific problem it doesn't have would cost trust for no reason.
Before you close the answer
Why this works
Tests whether you can sequence a real roadmap by what's actually hardest to undo, under the pressure of an interviewer naming the product live and asking you to defend it on the spot.
Follow-up traps
"Leadership wants gas furnace coverage before winter. What do you tell them?" Response: show them the callback-rate chart from the earlier miss; a six-to-eight-week delay costs less than a repeat of a carbon monoxide incident reaching a news story.
"Isn't a separate safety bar for every equipment type going to slow the whole roadmap down forever?" Response: no, it's a one-time cost per equipment type, paid once when that type is new, not a recurring tax on every future feature.
If pressed
The safety bar that followed required gas furnace diagnoses touching a heat exchanger or combustion component to always route to mandatory human verification, regardless of the model's own confidence score, a hard rule with no override.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
