How do you build the ROI case for an AI feature before it ships?

Every month, about 133 of Brackenfell Dental Group's booked appointments turned into empty chairs. Recallwise, Stonewick Health's scheduling and reminder tool, was already running there, sending the same reminder to every patient, 48 hours out, no matter how that patient usually behaved. Osareme Duskmere owns Recallwise's roadmap at Stonewick, and before asking for a full quarter of two engineers' time to build something smarter, she had to answer one question with real numbers: would it actually pay for itself.

How to answer this, stage by stage

Nobody is grading whether Osareme can multiply. They're grading whether she'll hand over one confident total, or show which number inside it she trusts least, and go fix that one before spending real engineering time.

Let's learn

What does it actually take to know, before you build it, whether a feature will pay for itself?

Recallwise is Stonewick Health's tool for dental practices. It sends appointment reminders, answers routine scheduling texts, and offers a new time when someone says they can't make it, without a front-desk person picking up the phone first.

Before the new feature, Brackenfell's flagship location sent one reminder, at one time, to every patient, no matter how that patient usually behaved.

Before Predictive Reminder Timing, every one of Brackenfell's 1,400 monthly appointments got the same text, 48 hours out. No-show rate: 9.5 percent, about 133 empty chairs a month. An existing feature, a same-day waitlist, already backfilled a quarter of those slots. The rest just sat empty.

Predictive Reminder Timing learns, per patient, when they actually tend to read a message and reply, and offers a new slot automatically if they say they can't make it. Stonewick ran it as a pilot at Brackenfell's flagship location for six weeks. No-show rate fell to 6.6 percent, about 92 a month. Counting the same waitlist backfill, that's 31 real slots recovered a month, at roughly $172 a slot, Brackenfell's own blended rate across hygiene and restorative visits.

Here's the turn. A feature that recovers real money for a dental practice isn't automatically a good bet for the company that builds it. Osareme doesn't get to keep Brackenfell's five thousand dollars. She gets whatever price practices will pay for the feature, minus whatever it costs Stonewick to run it, forever, across every practice that turns it on. That's a completely different sum, built from completely different numbers, and most of them didn't exist yet.

One step in the middle, the confidence check, decides whether a reschedule costs a text message or a live phone call.

So Osareme wrote the equation out loud, the way BOUND asks for, before trusting a single figure inside it.

Six numbers feed one total. Only one of them turned out to be the risky one.

Money in: how many of Stonewick's 340 practices turn the feature on, times a $60 monthly add-on fee, times twelve. The price wasn't a guess. Brackenfell's own recovered revenue was over eighty times that fee, so $60 was always going to be an easy yes for a practice that had seen the pilot's numbers.

Money out: how many appointments run through the model across the whole install base, average 1,100 a month per practice, times what one reminder cycle actually costs to run, times twelve. Built up from the ground: a base text message, $0.008, already a known cost. A small model call to decide the best time and channel, $0.004. And an escalation cost, a live AI voice call, for the reschedules a text alone can't close or the model isn't confident enough to book on its own, at $0.18 a call.

That last piece is where the case actually lived or died, and Osareme's first pass used the only number she had: the pilot's own escalation rate, 10 percent of appointments. That gave a running cost of about three cents an appointment, and a build cost of $95,000, two engineers for one quarter.

What that costs at its worst: if Stonewick greenlights the full quarter and ships this to the whole install base on a running-cost number that's actually wrong, the mistake doesn't show up on day one. It shows up a year later, buried inside a blended margin line, the same way a wrong pricing assumption hid inside Nettlebright's account for months in a different case. By then the quarter is already spent and the feature is already live everywhere.

What I would leave alone: the no-show reduction itself, 9.5 percent down to 6.6, and the base text-message cost, $0.008. Both held steady through everything that came next. The risk was never in the value side of the equation. It was hiding in one line of the cost side.

The lesson: an estimate is only as trustworthy as its shakiest number. Find the shakiest one before you trust the total, not after the quarter is already spent.

Now here is the same thing as a story

Read the short version above when you're in the room. Read this one when you want to feel why a case that sailed through review almost got built on a number nobody had actually checked.

Osareme Duskmere has never once brought a build request to roadmap review without a spreadsheet behind it. Three years into owning Recallwise's roadmap, she still builds the numbers by hand before trusting anyone else's summary of them, her own included.

The pilot at Brackenfell's flagship location ran clean. Six weeks, one location, no-show rate down from 9.5 to 6.6 percent, real dollars recovered, Bertram Aldercress, Brackenfell's COO, telling her outright he'd pay double the fee she eventually asked for. Osareme built her first pass of the case the same week the pilot closed. Adoption between 35 and 55 percent, a fee that undercut the value ten times over, a build cost of $95,000. The payback came out between nineteen and thirty months. She sent it to the roadmap deck.

It looked airtight in three quiet beats. Beat one: the first pass used the pilot's own escalation rate, 10 percent, because it was the only number that existed for a feature that hadn't shipped yet, and nobody in the review room asked where it came from. Beat two: a colleague flagged, almost as an aside, that six weeks at one location was a small sample to hang a company-wide number on. Osareme noted it, agreed in principle, and moved on, because the direction of the whole case was so clearly positive that the caveat felt like a formality. Beat three: the case sailed through the roadmap review with barely a follow-up question. Everyone liked the story. Nobody asked to see the escalation number's own history.

No single bad decision. A number nobody checked, sitting quietly inside a case everyone liked.

The trigger wasn't a dashboard alarm or an angry email. It was a data engineer, two days before the build was meant to start, mentioning almost in passing that the base reminder feature's own escalation rate, the one already running in production across all 340 practices, had been sitting at 16 percent for six months. Not 10. Osareme had never pulled it, because the case wasn't about the base feature. It just hadn't occurred to her that a close cousin's real number was one query away the whole time.

One of these numbers had six weeks behind it. The other had six months, and Osareme had never asked it a question.

She rebuilt the case that afternoon with the real number. Running cost per appointment moved from three cents to about four. Not a dramatic jump on paper. But at every adoption level, the payback slid past two years, and at the low end, it moved out past four.

Run it again, with the real number in hand from the start. Osareme doesn't take the case back to review as a "build it" or "don't." She takes it back with a third option: extend the pilot two more weeks, three more locations, about $4,000, and measure Predictive Reminder Timing's own escalation rate directly, instead of borrowing the base feature's number one more time. If the real rate lands close to 10 percent, the original case holds and the quarter gets approved next sprint. If it lands near 16, the fee moves to somewhere around $75 to $80, or the first rollout goes only to the practices with the worst no-show rates today, where the value per practice is highest and the model has the least room to guess wrong.

One version of this case would have committed a full quarter of engineering time off a number that had six weeks of evidence behind it, sitting three rows down in a spreadsheet from a number with six months. The other spent $4,000 and two weeks finding out which one was real before the bigger bet got made.

What Osareme would tell herself, back in that first roadmap review: the case didn't fail because the estimate was wrong. It almost failed because nobody, herself included, had asked out loud which number in it was the one doing all the work.

BOUND, or how not to greenlight a quarter off a borrowed number

Not a way to dress up a spreadsheet in five letters. BOUND is what stops an estimate from being a confident total with no visible seams, by forcing every number in it to say where it came from and how much it's actually trusted.

The same threshold that protects a patient from a wrong booking is also the lever that decides how expensive the feature is to run.

Three things worth stating directly, since this is where the real judgment sits. Stonewick considered, and rejected, bundling Predictive Reminder Timing into the base subscription for free, shipping it to all 340 practices at once with no opt-in and no fee. It lost, because the running cost scales with every appointment on the platform whether or not that practice had a no-show problem worth solving, guaranteeing the case could never recoup its build cost inside any reasonable window. The AI-specific failure worth naming is a confidently wrong auto-reschedule: the model books a new slot with the wrong provider, the wrong insurance coverage, or the wrong visit length, and the patient only finds out at the counter. The guardrail is the 90 percent confidence threshold itself, the bar the model has to clear before it acts alone, most of the time, by design, not a promise that it's always right. And the trade-off is real and accepted on purpose: keeping that threshold at 90 percent, instead of lower, means more reschedules escalate to a live call, which is exactly what makes the running cost higher than the naive case assumed. Stonewick chose that cost on purpose, because a wrongly auto-booked appointment costs more in a frustrated patient and a support ticket than the extra escalation calls ever save.

And if you want to be sure it really works, try it somewhere else

Same five letters, an industrial laundry instead of a dental office, and this time the swing factor isn't a dollar figure at all. It's whether the model's own catch rate can be trusted off the number of real events it's actually seen.

Driftguard is Grishwold Systems' tool for commercial laundries: it reads vibration data off a washer-extractor's bearings and flags a likely failure five to ten days before it happens, instead of the machine going down mid-shift with no warning. Copperfen Linen Services runs 24 of these machines across three plants, and had been losing about $84,000 a year to unplanned bearing failures, six a year, fleet-wide, at roughly $14,000 each in lost throughput, expedited parts, and overtime to catch up.

Driftguard ran on six of Copperfen's machines for four months. It flagged three real events; two were confirmed bearing wear caught in time, one was a false alarm. Grishwold's engineer, sizing the build case, used an industry-benchmark catch rate, 70 to 85 percent, rather than the pilot's own small-sample 100 percent, because two real events aren't enough to trust a rate at all.

Not every machine in the fleet carries the same risk. The washer-extractors, run hardest and costliest to lose, are where the model's judgment actually pays for itself.

At a 70 percent catch rate: 4.2 failures caught a year, worth $58,800, against roughly $756 in false-alarm inspections and $3,600 a year to run the sensors and the model. Net, about $54,444 a year, against a one-time build and sensor cost of $20,400. Payback: about four and a half months. At 85 percent: payback closer to three and a half months. Both fast, both a clear yes.

Same method, different lever: at Brackenfell, the number that swung the whole case was a dollar figure, the escalation rate. At Copperfen, the dollar swing barely matters, four and a half months versus three and a half either way clears any reasonable bar. What actually matters here is whether the catch rate can be trusted at all off two real failures. A model's own precision and recall aren't safe to trust from a handful of events, no matter how good they look, and that's the number Grishwold would go get more of before scaling to the rest of the fleet, not the dollar total.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to the equation: failures caught, times cost avoided, minus false alarms and running cost, minus the one-time build.
Cost: no budget this quarter to extend either pilot. Pull whatever adjacent production number already exists, the way the base reminder feature's real escalation rate was already sitting in a dashboard, before assuming a small pilot's guess is the only number available.
The model got better, for real: say Driftguard's catch rate turns out to be 95 percent instead of 70 to 85. The direction step barely moves, because the swing was never really about the catch rate's size, it was about whether two events were enough to trust any number at all.

Where people run it wrong.
They let a small pilot's number stand in for a rate that needs a much bigger sample before anyone should trust it.
They lead with one confident total instead of a range, and let the room assume more precision than the estimate actually earned.
They skip the sanity check against the last comparable feature, and miss that a case is quietly worse than the company's own track record.

How to use it live. Ask yourself, out loud if you have to: "which number in this equation have I actually tested, and which one am I just hoping holds?" That question alone is usually the difference between an estimate and a guess with a spreadsheet attached.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0