Tear down a product I name, live, in ten minutes.

Kestrel Ledger flags customers likely to cancel their phone plan, so a support rep can call before they leave. Idris Kavanjian runs support operations at Basalt Meridian Communications and was handed exactly this product, live, in a ten-minute stakeholder review, no prep time.

How to answer this, live, minute by minute

Six moves, budgeted against an actual ten-minute clock. This is the one skill that matters most when someone hands you a product cold.

Let's learn

Kestrel Ledger scans a customer's usage pattern, billing history, and support-ticket history, and flags the ones it thinks are about to cancel, so a rep can call before they do.

Before Kestrel Ledger, Basalt Meridian's retention team called customers only after they'd already started a cancellation request, catching roughly 15% of them with a last-minute offer.

With Kestrel Ledger, reps now call proactively, before any cancellation request starts, based on a churn-risk flag alone.

The turn: the 89% figure sounds like the whole story, but it only measures whether the flag matched a customer who later left. It says nothing about whether the phone call changed anything.

At its worst: Basalt Meridian signs a two-year contract on the strength of "89% accuracy," and the actual save rate turns out low enough that the tool costs more in outreach and licensing than it returns in kept customers, discovered only at renewal.

What I would leave alone: Kestrel Ledger's underlying flagging model itself is probably fine. The 89% figure, whatever it actually measures, likely reflects real work. The gap is entirely in what got reported, not necessarily in what got built.

The lesson: a live teardown isn't about producing a verdict in ten minutes. It's about finding the one unmeasured number the whole decision quietly depends on, and saying so plainly before the meeting ends.

Now here is the same thing as a story

The minute-by-minute version above is what you'd actually say in the room. Read this one for how the real ten minutes played out.

Idris has run support operations for six years and has sat through more vendor pitches than he can count, most of them longer than ten minutes and worth less than five.

The flagging step gets all the marketing attention. The step after it, whether the call actually works, gets none.

The meeting opened with a slide: "89% flag accuracy, validated across 40,000 customer records." Everyone in the room, including Idris for the first thirty seconds, read that as "this product works."

Five checkpoints across ten real minutes, not a leisurely hour-long teardown.

The trigger was one word in the slide's own footnote: "accuracy measured against churn outcome, independent of outreach." Idris caught it on the second read and asked the vendor directly: "so this number doesn't include whether the call worked?"

Only one of these four branches is a genuine save. The vendor's 89% doesn't distinguish between any of them.

The vendor's answer confirmed it: no, the 89% was purely about whether the flagged customer actually churned, not whether the rep's call changed the outcome. Nobody at Basalt Meridian had asked for the save rate before that day.

Not every flagged customer is worth calling, and not every call is worth the same amount, a distinction the 89% number can't make.

With the range worked out live, from an 18,000-dollar loss at a conservative save rate to a 62,000-dollar gain at an optimistic one, the room agreed to a thirty-day, single-region pilot instead of signing the full contract on the spot.

The vendor's slide answered zero of these four questions. All four had to be asked live, in the room.

The old habit was to read a headline accuracy number as the whole verdict on a product. The new one asks, in the first minute, what that number is actually measuring, before any range gets built at all.

Idris used to let a clean-sounding percentage carry more weight than it earned, since pushing back live, in front of a room, felt like it cost more time than it was worth. It took one footnote, caught on a second read with the clock running, to see that ten minutes spent finding the right question beats an hour spent trusting the wrong answer.

BOUND, in one screenNot a leisurely estimate. BOUND is the shape a good teardown takes when the clock is actually running.

Five letters, and under a real clock, direction is the one that has to land before time runs out.

The recap, one line per letter: break it down is the equation with the save rate named as missing, own numbers is the 10-30% industry-range assumption, use a range is the eighteen-thousand-loss-to-sixty-two-thousand-gain spread, nail the sanity check is comparing it to one rep's salary, and direction is the save rate as the number worth a real pilot.

And if you want to be sure it really works, try it somewhere elseSame five letters, a school district's enrollment-forecasting tool, ten minutes, a different clock entirely.

Tindersong forecasts which students are likely to withdraw mid-year, so a school counselor can intervene early. Marguerite Osafo, a district administrator, gets ten minutes to tear it down live at a budget review.

Mapped onto BOUND: break it down is net value equals students retained times per-student funding, minus counselor time spent on flagged students, minus students who withdraw despite intervention. Own numbers: Tindersong's vendor reports "84% withdrawal-prediction accuracy" but no retention rate after counselor contact, so Marguerite assumes a 15-25% retention range based on comparable early-intervention programs. Use a range: net value spans a modest loss to a genuine gain, same shape as Kestrel Ledger's spread. Nail the sanity check: at the low end, the tool costs more in counselor hours than the per-student funding it protects. Direction: the retention-after-contact rate, not the prediction accuracy, is what swings this the most, and it's the one thing a single semester's pilot could measure directly.

Swap "save rate" for "retention-after-contact rate" and the exact same four missing pieces show up in a school district's pitch too.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds, not ten minutes. Say "89% is flag accuracy, not save rate, ask for the save rate before anything else," and stop.
Cost: there's no time in the actual meeting to build the full range live. Say so honestly, and commit to one number to go verify afterward, rather than pretending to finish an estimate you don't have inputs for.
The model gets better, for real: if Kestrel Ledger's flag accuracy improves to 95% next quarter, that's still not an answer to the save-rate question, a more accurate flag on a customer nobody successfully calls back is still zero saved revenue.

Where people run it wrong.
They accept the vendor's headline number as the whole answer instead of asking what it actually measures.
They try to build one precise final number under time pressure instead of giving an honest range.
They spend all ten minutes debating the model's accuracy instead of finding the one input nobody has measured yet.

How to use it live. When someone hands you a product to tear down on the clock, spend your first sixty seconds finding the one number the whole pitch quietly assumes but never states. Everything else you say afterward should point back at that number.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0