ConceptIntermediateAI Opportunity & Model Strategy / Roadmapping under model uncertainty / #2

Describe the difference between a capability bet and a feature commitment.

PICK the auto-apply button that fixed the code and quietly stopped teaching anyone why

Two roadmap lines can look almost identical and mean completely different things: one is a promise about what the product does today, the other is a bet on what the model will be able to do later. Verdreth builds an AI code-review assistant for engineering teams. Priyanka Sood is Head of Product, and the question in front of her is why her team keeps confusing the two.

The direct answer
A feature commitment is a promise about a fixed outcome, built to work with the model exactly as it is today. A capability bet is a promise about a threshold the model hasn't crossed yet, and shipping a date for it means shipping a bet, not a result. The dangerous move isn't picking the wrong one. It's labeling a capability bet as if it were a feature commitment, so nobody ever revisits whether the threshold actually got crossed.
Do this, in order
  1. Name which promise you're actually making before you name a date.Why: a fixed-outcome promise and a threshold bet need completely different commitments from you.
  2. Find where the hidden cost sits: not in the slipped date, in the mislabeled one.Why: a late capability bet is visible and cheap to absorb. A capability bet disguised as a shipped feature is invisible and expensive.
  3. Write the kill criteria for every capability bet before you start it.Why: without a stated bar, "almost there" can stretch for years and nobody can call it.
  4. Rebuild the pause a merged action removed.Why: the actual product decision here is putting back the moment where reasoning got explained, not just tracking two kinds of promise on a slide.
  5. Watch the number that would look fine right up until it doesn't.Why: a skill quietly eroding in the people around the tool won't show up in any accuracy dashboard.

How to answer this, stage by stage

Nobody is scoring whether you can define two terms. They're scoring whether you can tell, on sight, which promise a given roadmap line actually is.

Stage 1
Give the definition in one breath each
Say it like this
"A feature commitment works with the model as it stands right now. A capability bet only works once the model clears a bar it hasn't cleared yet. Same roadmap slide, two completely different kinds of promise."
Why this works
Answers the literal question first, before any story or example.
Stage 2
Ground it in one real product
Say it like this
"Let's use Verdreth, an AI code-review tool. 'Flag common style issues' is a feature commitment. 'Explain a subtle architecture tradeoff the way a senior engineer would' is a capability bet."
Why this works
Keeps two abstract terms from floating away from anything real.
Stage 3
Say the structure out loud
Say it like this
"I'll run this as PICK. Position, my actual pick. Impact, who feels each kind of wrong. Cost asymmetry, which error is cheap and which is hidden. Kill criteria, what evidence would make me shelve the bet."
Why this works
Shows a repeatable way to sort any roadmap line, not just this one.
Stage 4
Name the real asymmetry
Say it like this
"A capability bet that slips is cheap. Everyone sees the date move. A feature commitment quietly relabeled as 'the model will get smarter here soon' is expensive, because it ships on time, looks fine in the demo, and nobody ever comes back to check whether it actually got smarter."
Why this works
This is the actual judgment call the question is testing, stated plainly.
Stage 5
Prove it with the compressed failure
Say it like this
"Verdreth merged 'flag the issue' and 'auto-apply the fix' into one button, marketed as a capability that would keep getting smarter. It never got revisited. A senior engineer's weekly review time dropped from four hours to forty-five minutes, and her two juniors quietly stopped learning why fixes worked at all."
Why this works
Compresses the whole failure into one line an interviewer can picture.
Stage 6
State the kill criteria, then close
Say it like this
"Any capability bet needs a named threshold and a recheck date. If Verdreth's explanation quality hasn't moved in two eval cycles, that's the signal to stop calling it a capability bet and ship the narrower, honest feature instead."
Why this works
Closes on the one line that separates a real decision from a hope.

Let's learn

Here is what a code-review tool looks like when a promise about tomorrow's model quietly gets treated like a fact about today's.

Before Verdreth, Dana Okafor, a six-year backend engineer, reviewed every pull request from her two junior engineers by hand. About forty minutes a PR, six PRs a week, roughly four hours she spent walking through her reasoning line by line in the comments. With Verdreth, a bot flags likely issues in seconds and, for common ones, offers a one-click "auto-apply fix." Dana's review time dropped to about forty-five minutes a week for the same six PRs.

Hand sketched comparison titled Two different promises. Left panel, a box icon labeled Feature commitment, caption a fixed outcome built for the model as it is today. Right panel, a gauge icon labeled Capability bet, caption a threshold the model hasn't crossed yet, shown in a different color.
Same roadmap slide, two different contracts. Only one of them is safe to put a date on.

Here's the turn: the extra mistakes that occasionally slip through auto-apply are not the real problem. The real problem is what auto-apply removed on the way to being fast: the pause where Dana used to explain, in words, why a fix worked. That pause was never on anyone's roadmap. It just used to exist, because two separate clicks used to be two separate moments.

Dana's weekly code-review time, before Verdreth, after auto-apply, and after the fix
4h 2h 0 4h 00 Before Verdreth 0h 45 After auto-apply 1h 30 After the fix
Review time never went back to four hours, and it didn't need to. It only needed to go back up enough to hold a real explanation.
Junior engineers' independent bug-catch rate, month over month after auto-apply shipped
40% 20% 0 M1: 38% M3: 24% M6: 14%
Nothing in Verdreth's own accuracy numbers moved during these six months. This is the number that was actually falling the whole time.

At its worst, this looks like a product that got faster and a team that quietly got weaker in a way no dashboard would show for months.

The choice I would take back Verdreth merged "flag the issue" and "apply the fix" into a single button, marketed under the same capability-bet promise as the flagging itself: that explanations would keep getting sharper release over release. That made sense when the two actions were new and either one alone felt like friction. It stopped making sense once "apply" started running ahead of any explanation ever showing up.

What I would leave alone: the flagging step itself. Dana still wants a fast first pass on obvious issues, and reviewing every trivial style flag by hand would be exactly the kind of caution that helps nobody.

The lesson: a capability bet you never wrote a threshold for isn't a bet. It's a hope wearing a roadmap's clothes, and it will sit on the roadmap forever looking done.

Now here is the same thing as a story

The short version above is what you'd say scoping this live in an interview. Read this one for how one question from a new hire undid eight months of assuming.

Dana Okafor has spent six years writing backend services, the last three of them mentoring whoever joins her small team. She can read a diff and tell, in about ten seconds, whether the person who wrote it understood the tradeoff they were making or got lucky.

For Verdreth's first eight months on her team, the tool was a quiet, useful pre-filter. It flagged the obvious stuff, an unclosed transaction, a missing null check, and Dana still wrote out the reasoning herself in the PR comments, same as always. Review time barely moved. Nobody complained, because nothing had really changed yet.

Hand sketched metaphor scene titled Where the reasoning used to live. Left, a person icon labeled Dana reviews, caption walks the fix through line by line. Right, a gauge icon labeled Auto-apply, caption fixes it, explains nothing, shown in a different color.
Same fix, two very different amounts of understanding left behind afterward.

Then Verdreth shipped auto-apply, pitched internally and externally as a capability that would keep getting more precise with every release, the same open-ended promise the flagging feature had always carried. One click now flagged and fixed a whole class of common issues at once. Review time for those PRs dropped from ten minutes each to about ninety seconds. Dana, like everyone on her team, started clicking through them fast. There was, after all, always a next PR waiting.

Knowledge spark: what's a capability bet, really? It's a promise that only comes true once the model gets better at something specific. Not "the tool works." "The tool will be able to explain reasoning as well as a senior engineer, once it clears some bar it hasn't cleared yet." Until someone names that bar, the promise can't be checked, only believed.

Nobody at Verdreth decided, on any single day, to stop teaching junior engineers. The auto-apply button applied to everyone's PRs equally. But the two things that used to happen in sequence, flag then explain, had been merged into one thing: flag-and-fix, no explanation required. The pause where reasoning used to live simply had nowhere left to go.

Hand sketched flow diagram titled Where the explanation step went missing. Four boxes: PR opened. Verdreth flags it. Auto-apply fix, emphasized in a different color. PR merged.
Three steps look the same as before. The fourth one used to be a person explaining something out loud.
The auto-apply button didn't cost Dana ninety seconds of review time. It cost her team the two years of judgment her juniors used to build by watching her reasoning happen in public, one PR at a time.

It took a new hire, five weeks in, asking Dana in a stand-up why a particular fix had been applied the way it had, for anyone to notice. Dana didn't know either. She'd stopped reading them in detail months earlier, same as everyone else. That was the moment the merged button stopped being a convenience and started being a capability bet nobody had ever actually verified.

Hand sketched timeline titled How the mentoring habit thinned, third milestone emphasized. Four milestones: Verdreth launches, a small pre filter, easy to override. Auto-apply ships, flag and fix merge into one click. A new hire asks why, nobody can say what the fix actually did. Review time reclaimed, the explanation step gets rebuilt.
Each step made sense alone. Together, they quietly removed the one thing a junior engineer actually needed to see.

What I'd tell myself, watching that new hire's question land in the room: the auto-apply button was never wrong to exist. It was wrong to be sold, internally, as a capability that was already delivering more than it had ever been asked to prove.

PICK, the pick that survives being pushed on

P
Position. The pick, stated first.
Treat a feature commitment and a capability bet as two different contracts, and never let a bet get labeled with a shipped-feature's confidence.
Naming the pick before any reasoning is what lets an interviewer test whether you'll commit.
I
Impact. Who feels each kind of wrong.
A mislabeled feature commitment costs the engineering team quiet technical debt. A mislabeled capability bet costs sales a promise that may not exist yet.
Naming both sides in real units, not abstractions, is what makes the tradeoff checkable.
C
Cost asymmetry. Which error is cheap, which is hidden.
A late capability bet is visible and gets absorbed. A capability bet dressed as a shipped feature is invisible, and it decays into the exact "fake AI" a user eventually resents.
This is the hardest step, and the one the whole answer turns on.
K
Kill criteria. What would flip the pick.
If explanation quality hasn't moved in two eval cycles, or closing the gap needs an architecture change nobody has announced, shelve the bet and ship the narrower feature.
A stated bar is what separates a real bet from a hope with a deadline.

The recap, one line per letter: position is naming the two contracts before naming any date, impact is the engineering debt on one side and the sales promise on the other, cost asymmetry is the mislabeled bet being the expensive one because it's invisible, and kill criteria is a stated threshold and recheck date that lets you actually call time on a bet that isn't paying off.

And if you want to be sure it really works, try it somewhere elseSame four letters, a veterinary clinic's scheduling assistant instead of a code-review tool. Different flip family entirely, the same undated bet.

Oakendell Veterinary Partners runs an AI assistant that suggests appointment slots for incoming calls. Mapped onto PICK: position is the clinic treating "suggest a routine checkup slot" as a feature commitment and "suggest a slot for an ambiguous, possibly-urgent case" as a capability bet still waiting on a threshold. Impact lands on Hollis Bramwell, the clinic manager, who absorbs a wasted ten-minute hold if a routine suggestion is off, versus a family whose sick animal waits two extra days because an urgent case got slotted like a routine one. Cost asymmetry favors the same shape as Verdreth's: a slow routine suggestion is cheap and visible, a wrongly-confident urgent suggestion is hidden until the family calls back worried. Kill criteria is a stated miss-rate on ambiguous cases; above five in a hundred, urgent-sounding calls route to a human first, no exceptions. The flip here is abandonment, not delegation: once Hollis's front-desk staff got burned twice by confident-sounding suggestions on ambiguous calls, they quietly stopped opening the suggested-slot panel at all, for every call, including the easy ones the tool actually handled fine.

Hand sketched labeled parts diagram titled What makes a capability bet real, not just a hope. A gauge icon at the center labeled Capability Bet, with four labeled callouts: A named threshold, A recheck date, A fallback feature, An owner who revisits it.
The same four parts turn "it'll get smarter" into something you can actually check, in a clinic or a code-review tool alike.
Hand sketched quadrant titled Oakendell's scheduling suggestions, plotted. X axis model confidence, low to high. Y axis cost if wrong, cheap to expensive. Routine checkup slot placed high confidence, cheap. Walk-in emergency slot placed low confidence, expensive. Follow-up reminder placed high confidence, moderately cheap. Multi-pet booking placed medium confidence, moderate cost.
The quadrant makes the same point the code-review story did: it's not about the tool's overall accuracy, it's about which quadrant a given call falls into.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "never let a bet on tomorrow's model borrow a shipped feature's confidence," and stop.
Cost: no time to build a recheck process this quarter. Say so honestly, and put a manual calendar reminder on every open capability bet rather than let it drift with no owner at all.
The model gets better, for real: if Verdreth's explanation quality genuinely clears the threshold next quarter, that's the bet paying off as designed, and the honest move is converting it into a real feature commitment, not treating "it might still improve more" as a reason to keep dodging the question.

Where people run it wrong.
They ship a capability bet with no threshold, so "almost there" can mean anything for as long as anyone likes.
They treat a slipped date as the only real cost, and miss the cheaper, quieter one sitting behind a feature that shipped on time and never improved.
They let one team's optimism about the model set the whole roadmap's confidence, instead of naming the bar out loud where everyone can see it.

How to use it live. The moment a roadmap line sounds like both a promise and a hope in the same sentence, ask which one it actually is. If nobody can name the threshold, it's a hope, and it needs to say so.

Flashcards (tap any card to flip it)

1 · THE FLIP FAMILY
What flip family is this?
Tap to flip
ANSWER
Delegation flip: Dana handed first-pass review down to Verdreth's auto-apply button, and once a new hire's question exposed the gap, she had to reclaim the explanation step herself.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Dana Okafor, a six-year backend engineer who mentors two junior engineers through her PR review comments.
3 · THE HABIT
What did Dana stop doing once auto-apply shipped?
Tap to flip
ANSWER
She stopped writing out her reasoning in PR comments, since the merged flag-and-fix button no longer left a natural place for an explanation to go.
4 · THE DIFFERENCE, IN THIS STORY
What separates a feature commitment from a capability bet here?
Tap to flip
ANSWER
Flagging obvious issues is a feature commitment, built for today's model. Explaining reasoning the way a senior engineer would is a capability bet, resting on a threshold Verdreth had never actually crossed.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Merging "flag the issue" and "apply the fix" into one button, sold under the same open-ended capability promise as the flagging feature, a call that made sense when both actions still felt like separate friction.
6 · THE NUMBER
Fill in the blank: Dana's weekly review time dropped from 4 hours to ___ minutes after auto-apply shipped.
Tap to flip
ANSWER
45 minutes, before settling at 1 hour 30 minutes once the explanation step got rebuilt.
7 · THE REPLAY
Same new hire, same question, but the kill criteria already written. What changes?
Tap to flip
ANSWER
Auto-apply still merges flag and fix, but a monthly recheck against the named threshold would have already flagged that explanation quality hadn't moved, forcing the honest conversation months before a new hire had to ask it out loud.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this same question again for a different product, with a different flip family. Which product, and which family?
Tap to flip
ANSWER
Oakendell Veterinary Partners' scheduling assistant. The flip is abandonment: front-desk staff quietly stopped opening the suggested-slot panel at all after two bad calls on ambiguous cases.

Check yourself Score: 0 / 0

Multiple choice
1. Which of these is the actual definition of a capability bet, according to this answer?
  • A. Any AI feature that hasn't shipped yet.
  • B. A promise that depends on the model crossing a threshold it hasn't crossed yet.
  • C. A feature that is more expensive to build than a normal one.
  • D. Any roadmap item without a fixed ship date.
Show hint
Look at the direct answer and the comparison diagram.
Show answer
B. A feature commitment works with the model as it is today. A capability bet only works once the model clears a bar it hasn't cleared yet.
True or false
2. True or false: this answer says the most expensive mistake is a capability bet that slips its date.
  • True
  • False
Show hint
Look at the "cost asymmetry" step.
Show answer
False. A slipped date is cheap and visible. The expensive mistake is a capability bet mislabeled as an already-shipped feature, since nobody ever revisits it.
Fill in the blank
3. Fill in the blank: after the fix, Dana's weekly review time settled at ___ hour(s) and 30 minutes.
Show hint
Look at the grouped bar chart.
Show answer
1 hour. Not back to the original 4 hours, just enough to hold a real explanation again.
Short answer, name the reversal
4. What old decision does this answer take back, and why did it make sense when it was first made?
Show hint
Look at "the choice I would take back."
Show answer
Model answer: Merging flag-and-fix into one auto-apply button, sold under the same open-ended capability promise as the flagging feature. It made sense when both actions still felt like separate friction to remove.
Short answer, where it wouldn't matter
5. Name a part of Verdreth where this tension between the two kinds of promise genuinely doesn't apply.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: The flagging step itself. It's a plain feature commitment, working fine on today's model, with no threshold anyone needs to wait on.
Short answer, apply it yourself
6. Think of a product you use that promised something would "get smarter over time." Can you tell whether that promise was ever actually checked against a stated bar?
Show hint
Think of an app's recommendation feature, an autocorrect, or a smart-reply suggestion that never seemed to change.
Show answer
Model answer: A photo app's "smart album" sorting that was announced as improving with every update, with no visible way to tell if it ever actually did.
Before you close the answer
Why this works
Tests whether you'll treat "AI feature" as one bucket, or split it into two contracts with genuinely different risk, and whether you can say which one a given roadmap line actually is.
Follow-up traps
"Isn't this just semantics, since both eventually ship something?" Response: no, because only one of them can be verified against a stated bar. A feature commitment either works today or it doesn't; a capability bet with no threshold can be called "almost there" forever.

"What if the model improves faster than expected and the bet resolves early?" Response: that's still a win for the kill-criteria discipline, since a stated threshold lets you convert the bet into a real feature commitment the moment it's actually crossed, not just when it feels close.
If pressed
The fix that followed set a specific bar: any capability bet on Verdreth's roadmap gets a named eval set, a numeric threshold, and a 90-day recheck date, or it doesn't get listed as a bet at all, just an idea.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more