What is the opportunity cost argument against a technically feasible AI feature?

Codeloom builds PatchPilot, an AI tool that leaves inline review comments on pull requests. Odalys Kettering, a senior engineer, spent a hackathon building a prototype for Smell Certification: a deeper feature that flags architectural smells and rewrites commit messages, using a bigger, slower model call per PR. It works. Marek Branwell is the PM deciding whether to greenlight the full build.

How to answer this, stage by stage

Nobody is scoring whether you can define opportunity cost. They're scoring whether you'd have caught this before five weeks disappeared into the more exciting option.

Let's learn

PatchPilot is a tool that reads a pull request and leaves inline review comments automatically, so a developer gets feedback on style and likely bugs before a human reviewer ever opens the diff.

Before any real comparison, this is how a feasible prototype usually wins a slot on the roadmap: by being the thing that already works, not the thing that pays back the most.

Before Marek built a real comparison habit, Codeloom greenlit features mostly on whether the prototype worked. Two quarters ago, a similarly scoped AI feature, automatic test-name suggestions, shipped after a good demo, cost about the same five weeks, and get used by fewer than 4 percent of developers today. Nobody had ever named what those five weeks could have paid for instead.

So Marek wrote the comparison down properly this time: same build weeks, same two engineers, and then he added what each option actually returns. The flake fix's return came straight from Codeloom's own CI logs, real reruns, real hours lost, not a guess.

Here's the part that isn't about the smell feature being bad at all. It genuinely works. The turn is that "works" was never the actual question on the table. The question was always going to be whether it beat the next best use of the same five weeks, and until Marek wrote the comparison out, nobody had actually asked that question.

At its worst, greenlighting the feasible option without this comparison costs Codeloom two things at once: five weeks that could have recovered real engineer time starting immediately, and a new $8,700-a-quarter tax that keeps compounding as long as the feature stays in production.

What I would leave alone: PatchPilot's core inline comment feature never gets this scrutiny anymore, it already has two years of usage data showing it saves real reviewer time every week. The opportunity-cost check is for new, unproven asks competing against a known alternative, not for a feature that's already proven its return.

The lesson: feasible and worth building are two different questions, and a good demo only ever answers the first one. Write the second one down, in real numbers, before the five weeks are spent.

Now here is the same thing as a story

The short version above is what you'd say out loud in the room. Read this one for what it actually felt like to watch a great demo lose an argument it never knew it was in.

Marek Branwell had shipped four features at Codeloom, and his instinct for what would delight developers was good. His instinct for what to say no to was newer, and less tested.

Odalys Kettering walked into his one-on-one with a laptop open. "Look at this," she said, and ran Smell Certification against a real repo. It flagged a genuine circular dependency nobody had caught in review. It rewrote a vague commit message into something a teammate could actually search for later. It was, honestly, impressive.

The three real parts of the estimate, before either of them get turned into a chart. Only one of the three ever shows up in a demo.

For a moment, Marek almost said yes right there, the way Codeloom usually decided these things. Then he remembered the CI dashboard he'd been avoiding all week: 38 hours of team time a week, lost to reruns on flaky tests, a number that had been sitting in plain sight for a month.

"It's genuinely good," he told Odalys. "Give me two days before I greenlight it. I want to compare it against something real, not just against nothing."

Not AI versus no AI. Two real, scoped uses of the same five weeks, one proven and waiting, one exciting and unproven.

The two days weren't glamorous. He pulled Codeloom's own CI logs and found the 38-hour figure was real and conservative if anything. He priced Smell Certification properly with Odalys: five weeks to build, plus a genuine, recurring inference and eval bill neither of them had put a number on before. He priced the flake fix the same way: five weeks to build, once, with engineers who'd already scoped the root causes.

He never had a fixed number in his head for when a feasible feature should lose. He had a feeling with exactly two settings: it clearly wins on the numbers, or it doesn't. The demo alone never flipped that feeling. The written comparison did, in about twenty minutes of arithmetic.

Same effort, plotted honestly. The gap is entirely in the vertical axis, and that gap is what the demo never showed.

Back in that first meeting, saying yes to a good demo wasn't an unreasonable habit. It had worked fine for smaller, cheaper features where being wrong cost little. It stopped being reasonable the moment two options of equal size and cost were both sitting on the table at once, one of them already proven.

Here's the replay: same prototype, same impressive demo, but with the comparison run before any decision instead of after the fact.

The fork that actually decided this, drawn out. The flake fix wins because it was scoped and known, not because the smell feature was bad.

With the comparison run first, the flake fix ships this quarter, developers get back real hours starting week six, and Smell Certification moves to next quarter's list with its real cost already on record, ready to compare against whatever else is waiting by then.

One version of this story spends $38,700 to fix a problem nobody had measured yet. The other spends $30,000 once and starts recovering real time the same quarter, because somebody finally wrote both numbers on the same page.

What I'd tell myself, back in that one-on-one: a good demo is proof the model can do the task. It was never proof this was the best five weeks we had.

BOUND: five moves for the sizing question that actually decides your roadmapNot a script for justifying a decision you already made. BOUND is what makes you write the comparison down before the demo gets to decide for you.

The recap, one line per letter: break it down separates "can we" from "should this win," own the numbers prices both real options honestly, use a range checks whether the call survives being wrong, nail the sanity check names the trap of shipping whatever's feasible, and direction is the actual rule for what to do about it.

And if you want to be sure it really works, try it somewhere elseSame five letters, a warehouse routing product instead of a code review tool. The judgment call this time is a feasible AI feature against a known equipment fix.

Thessaly Vann runs product at Loadwell, a warehouse routing platform. An engineer built a working prototype for FlowScout, an AI feature that reroutes pickers in real time based on predicted congestion. It works in simulation. Loadwell also has a known, unfixed problem: a specific conveyor sensor that misreads about 6 percent of scans, forcing manual recounts that cost the floor about 22 hours a week. Mapped onto BOUND: break it down means the simulation proves FlowScout can predict congestion, not that it beats fixing the sensor with the same engineering time. Own the numbers means pricing both: FlowScout at four weeks to build plus an ongoing inference and monitoring cost against warehouse WiFi reliability, the sensor fix at four weeks to build, once, with a real 22-hours-a-week recovery already measurable from the recount logs. Use a range means checking what happens if FlowScout's congestion predictions turn out to be only 70 percent reliable in a real warehouse instead of the simulation's 94 percent, which would need human override often enough to erode most of its value. Nail the sanity check means noticing that "the model is impressive" and "the model is the best use of four weeks" got treated as the same claim. Direction is the same rule again: fix the sensor first, this quarter, and re-scope FlowScout once real warehouse WiFi data exists to price its actual reliability.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Skip straight to "same weeks, name what else they could build, price both, let the number pick," and stop.
Cost: no time to pull real CI logs before the meeting. Say so honestly, use a conservative estimate, and flag that the real number needs confirming before the call is final.
The model got better, for real: say a future version of the smell detector needs far less eval upkeep. Rerun the comparison, because the ongoing tax shrinking is exactly the kind of change that should flip a close call.

Where people run it wrong.
They treat "it's technically possible" as the whole approval bar, skipping the comparison against any real alternative entirely.
They price the AI feature's build cost only, and forget the ongoing inference and eval bill that a rule based fix would never carry.
They let which option demoed better decide, instead of which option's real numbers actually win.

How to use it live. The moment an interviewer describes a feasible AI feature, don't ask whether it works. Ask what else those same engineering weeks could have built, and whether anyone ever wrote that comparison down. That question buys real thinking time, and it's usually exactly where the honest answer is hiding.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0