What is the right planning horizon for an AI product team, and why?

Verdalane Stream is a streaming service. Wren Ashbrook leads product for its personalization team, and the question on the table is a simple-sounding one that turns out not to be: how far out should an AI product team actually plan?

How to answer this, stage by stage

Nobody is scoring whether you can name a number of weeks. They're scoring whether you can tell which kind of roadmap item deserves which horizon, and why.

Let's learn

How far out should an AI product team actually promise something?

Verdalane Stream's personalization team wanted to ship mood-based playlists, generated from a listener's entire history instead of just their last few sessions. Before this project, the team ran on six-week cycles, and every cycle included a short check-in: does the thing we're building still assume what we think it assumes about the model underneath it?

Both errors cost something. Only one of them is expensive in a way nobody notices until it's too late to undo.

Leadership wanted something bigger: a twelve-month roadmap, timed to the fiscal year, promising whole-catalog mood playlists by Q4. The plan leaned on a vendor's preview API for a much larger context window, one big enough to hold a listener's full history in one pass. Early tests on a small internal sample looked excellent. Here's the turn: the twelve-month horizon wasn't wrong because it was long. It was wrong because it applied the same clock to a piece of infrastructure that would pay off regardless of the model, and a feature that only existed if one specific, unshipped capability arrived on schedule.

At its worst, a team can spend a year telling leadership a single confident number while quietly having no idea whether the one thing the whole plan depends on will exist on schedule.

What I would leave alone: I wouldn't touch the twelve-month horizon for the embeddings pipeline or the recommendation UI shell. Both of those pay off no matter which model the team ends up using, so there's no asymmetry to fix there.

The lesson: the right planning horizon isn't a company-wide setting. It's a property of each individual bet, and the bets that depend on one unproven capability need a much shorter leash than everything else on the same roadmap.

Now here is the same thing as a story

The short version above is what you'd say defending a roadmap review to leadership. Read this one for how a habit of checking quietly stopped, and what it took to notice.

Every Thursday, before the mood-playlist project started, Wren Ashbrook ran a fifteen-minute check-in with the data science lead: does anything we've built this cycle still assume what we think it assumes about the model. It was a small ritual. Almost nobody outside the team knew it happened.

A short horizon can absorb a wrong guess. A long one, tied to a single unproven assumption, cannot.

When the twelve-month mood-playlist roadmap got announced, the first two months were genuinely good. The vendor's preview API handled small test cases beautifully. Wren kept running the Thursday check-in, and it kept coming back clean: yes, the assumption still holds, yes, the timeline still looks right.

Somewhere around month five, the check-in stopped happening. Nobody decided to cancel it. It just kept confirming the same good news, and eventually the calendar invite got skipped once, then twice, then it quietly disappeared from the recurring schedule altogether.

Nothing dramatic happened at any single step. By the fourth, there was no habit left to catch the problem.

In month eight, a data scientist mentioned something in passing, in the middle of an unrelated planning meeting: "wait, are we still assuming that context-window SKU ships in general availability? I heard procurement's still negotiating the pricing tier." No alarm went off. Nobody had done anything wrong. It was one sentence, said almost as an aside.

Wren went looking for a fallback plan, something the team could ship on the current, smaller context window while the vendor sorted out the larger one. There wasn't one. The whole feature had been designed around the large window from day one, no chunked version, no interim design. Four months of engineering time had gone into a single, all-or-nothing bet.

None of the mood-playlist bet's four boxes were checked. All four of the embeddings pipeline's were.

The mood-playlist feature sits exactly where a bet needs a short leash: unproven, and the whole reason for the roadmap.

What I'd tell myself, watching Wren search for a fallback that was never built: the twelve-month number itself was never the mistake. The mistake was letting one unproven assumption ride on the same clock as everything else, with no smaller, provable piece underneath it.

PICK, choosing which horizon each promise deservesNot a scheduling preference. PICK is what tells you which roadmap item can survive a long promise and which can't.

The recap, one line per letter: position is splitting the roadmap by what depends on a model capability and what doesn't, impact is a few meetings against months of unshippable work, cost asymmetry is the hidden overcommit against the cheap and visible churn, and kill criteria is two stable release cycles and no single-vendor dependency before a bet earns a longer leash.

And if you want to be sure it really works, try it somewhere elseSame four letters, a container terminal instead of a streaming service. Different flip family entirely, the same one-bet roadmap.

Northquay Terminal committed to a nine-month rollout of an AI berth-scheduling optimizer, betting on a vendor's next-generation solver that promised to handle the terminal's full seasonal mix of vessel sizes. Mapped onto PICK: position is the same split, long horizon for the yard-mapping infrastructure underneath it, short horizon for the solver-dependent scheduling logic itself. Impact is felt by yard planners, who lose hours to bad berth assignments if the solver isn't ready. Cost asymmetry is the same shape: short replanning cycles are a minor annoyance, but a nine-month promise the whole terminal is counting on is a hidden, expensive bet if the solver slips. Kill criteria would have been the solver's benchmark holding flat across its last two releases, which it hadn't. The flip here is different: workaround, not over-trust. Rasheed Colby, a yard planner uncertain the nine-month bet would land on schedule, quietly started keeping his own spreadsheet predicting berth conflicts by hand, a shadow system nobody asked him to build and nobody at Northquay could see, support, or learn from.

The same question, asked before the nine-month commitment went out: what's actually been proven, and for how long.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "long horizon for what survives either model, short horizon for what depends on one, with a written kill line," and stop.
Cost: no time to split the roadmap into two tracks this quarter. Say so honestly, and at minimum tag every item with which kind of bet it is, so the risk is visible even before the process catches up.
The model gets better, for real: if the vendor's capability ships early and holds steady, the honest move is to extend the horizon on that item and say so, not treat every capability bet as permanently short-leashed out of habit.

Where people run it wrong.
They pick one horizon for the whole roadmap because it's simpler to plan around, instead of asking which items can survive being wrong.
They let a check-in habit fade quietly because it kept confirming good news, with nobody noticing the habit was the whole safety net.
They build the model-dependent bet with no smaller, provable interim version, so there's nothing to fall back on when the timeline slips.

How to use it live. The moment someone asks about planning horizons, ask yourself: which of these roadmap items would still be worth having built even if the model never improves again? Give those the long horizon. Give everything else the short one.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0