Build the roadmap review ritual you would run monthly for an AI team.

What would a monthly roadmap review for an AI team actually need to contain, on purpose, instead of whatever felt urgent that week? Wayfinder Housing Trust runs Haven Match, a tool that matches a person experiencing homelessness to an open shelter bed based on stated needs. Minh Tran, the nonprofit's Head of Product, built the ritual after two years of roadmap decisions made in hallway conversations.

How to answer this, stage by stage

Nobody is scoring whether you can list agenda items. They're scoring whether the agenda forces an actual rank, not a discussion.

Let's learn

What does a monthly roadmap review for an AI team actually need in it, so it decides things instead of just discussing them?

Haven Match reads a client's stated needs, family size, medical requirements, whether they have a pet, and ranks open shelter beds by best fit. Before the ritual, Wayfinder had no fixed process for deciding what to build next. Feature ideas came from caseworker complaints, funder suggestions, and whatever a board member had read about a new model that week. A change could ship because three people mentioned it in one week, or die because nobody happened to bring it up.

Five fixed steps, same order every month. The rank in step four is the part that actually does the deciding.

Here's the turn: the problem wasn't too many good ideas. It was that no idea ever competed against another idea on the same terms. A caseworker's complaint about pet-friendly filtering and a board member's excitement about a new model both got the same hallway-length hearing, and whichever one got said most recently usually won.

At its worst, this doesn't just waste meeting time. Wayfinder shipped a "smarter" ranking change in one quarter that nobody had backtested, and successful placements actually dropped for six weeks before anyone noticed, because the only measure anyone was tracking was whether the change had shipped, not whether it worked.

What I would leave alone: caseworkers can still flag urgent bugs any time, outside the monthly ritual. A ritual for prioritizing new work is not the same thing as an incident process, and forcing bug reports to wait a month would be its own kind of harm.

The lesson: a roadmap doesn't get chaotic because people care too much. It gets chaotic because nothing ever has to compete against anything else on the same terms. Build the ranking step, and caring stops being the same thing as winning.

Now here is the same thing as a story

The short version above is what you'd put in a one-page process doc. Read this one for how a shared tablet full of sticky-note ideas became the thing that finally forced a rank.

A shared tablet sat on the front desk at Wayfinder's main office, the one three staff members used to log client intakes. Somewhere along the way, it also became where people typed roadmap ideas whenever they thought of one: "add a pet-friendly filter," "what if we used the new model everyone's talking about," "a caseworker asked why it never explains its ranking."

Every idea on the tablet looked the same length. They were never the same size at all.

Minh Tran had run product at Wayfinder for three years, since before Haven Match existed. He was good at turning a caseworker's half-formed complaint into a clear feature. For a long while, that skill was enough. The tablet stayed short. Ideas got built, roughly in the order they arrived.

Then the tablet started filling up faster than anyone could act on it. Not because of one bad month. It built up slowly, over about a year, with no single moment anyone could point to. A funder mentioned a new model in a check-in call. A board member forwarded an article. A caseworker asked, again, why the tool didn't explain its ranking. Each idea got a hallway conversation, and whichever one had been said most recently felt the most urgent.

One quarter, a new ranking approach got built because it sounded genuinely smart in a meeting, and everyone wanted to try the new model behind it. Nobody backtested it against real placement history first, because there was no fixed step in the process that required it. Successful placements, clients still housed past 72 hours, dropped from 61 percent to 54 percent over six weeks before a caseworker flagged that something felt off.

The tablet never sorted ideas this way. Everything sat in one flat list, waiting its turn to be argued about.

So here is the decision I would take back: letting any idea reach Minh directly, at any time, without ever passing through a shared outcome number or a backtest. That was fine with three people and a handful of ideas a month. It stopped being fine once the tablet held more ideas than any one person could hold in their head at once.

The step that was missing sits second from the left, exactly where a hallway conversation used to stand in for it.

With the ritual in place, the replay runs differently. The same funder mention, the same board article, the same caseworker question all land on the tablet the same way. But now they wait for the first Tuesday of the month, where Minh reads Haven Match's one outcome number out loud, backtests each idea against real history, and forces a rank: advance, pause, or kill. The ranking model idea that once shipped untested now gets backtested first, and it either earns its place above the pet-friendly filter or it doesn't. And the thing I'd tell myself, if I could go back: I never needed to say no to more ideas. I needed a room where ideas had to prove themselves against each other before any one of them got to be the loudest.

ORDER, in one screen

The recap, one line per letter: outcome is successful placements past 72 hours, reversibility is telling a ranking tweak apart from removing the override, dependency is building the backtest pipeline before ranking anything against it, evidence is backtesting every idea before it's even discussed, and rank is stating the order and defending the top pick in one breath.

And if you want to be sure it really works, try it somewhere else

Thermwell Field Services runs DispatchLine, a tool that assigns HVAC technicians to service calls based on skill match, drive time, and part availability. Colette Marchetti, the dispatch lead, used to field roadmap ideas the same way Wayfinder once did, whoever radioed in an idea during the week got it raised at the next all-hands, in whatever order people remembered to bring it up. Mapped onto ORDER: outcome is on-time arrival rate, not dispatch-model accuracy alone. Reversibility separates a routing-weight adjustment, reversible in a day, from removing the dispatcher's manual override, which technicians would stop trusting the system enough to use again if pulled. Dependency means building a backtest against last month's actual routes before debating any new dispatch model. Evidence is running that backtest before the meeting, not during it. Rank forces a call on every item: advance, pause, or kill, defended in one sentence. The story underneath is different too, a delegation flip rather than the un-ranked hallway chaos at Wayfinder: once dispatch decisions started routing through a monthly ritual, the regional manager who used to personally override any technician's route he disagreed with had to let the ranked call stand instead of reclaiming it himself.

The same four weeks repeat every month, whether the roadmap is shelter beds or service trucks.

Same five parts as Haven Match's ritual, just measuring trucks instead of shelter beds.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "outcome number, what changed, backtest first, forced rank, name what stays," and stop.
Cost: no budget for a formal backtest pipeline yet. Say so honestly, and start with a manual spreadsheet check against last quarter's data, since a rough backtest beats no backtest at all.
The model gets better, for real: if a proposed change backtests as a clear win, that's the ritual working as intended, and the honest move is to rank it first and ship it fast, not slow it down out of habit.

Where people run it wrong.
They hold the meeting but skip the backtest, so the same hallway-persuasion dynamic just moves onto a calendar invite.
They rank by how exciting an idea sounds instead of by what breaks first if it's skipped or how hard it is to undo.
They never say out loud what's staying the same, so the team only ever hears about the roadmap when something's changing.

How to use it live. The moment someone asks you to build a review ritual, ask yourself: what in this agenda actually forces a rank, instead of just giving everyone a turn to talk? Build that part first, and the rest of the agenda follows.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0