Explain the difference between technical feasibility and product feasibility.

Say we build a tool that reads vibration and temperature off a building's cooling compressors and warns a field technician before one fails. Thermavane Controls calls theirs CircuitSense. Ingrid Pham-Solberg is the AI PM who proved it worked, then had to explain why proving it worked was not the same as it working.

How to answer this, stage by stage

Nobody is scoring whether you know the two phrases exist. They're scoring whether you can point at the exact place a technically sound model stopped mattering to a real person.

Let's learn

Say we build a tool that reads vibration and temperature sensors on a building's cooling compressors and warns a technician before one actually fails.

Before CircuitSense, Thermavane's technicians serviced every compressor on a fixed 90-day schedule regardless of its real condition, about 40 minutes a unit, catching a real problem mostly by luck. With CircuitSense, the model reads live sensor data and flags a specific unit as at risk, with the flag correct 89 percent of the time against Thermavane's own held-out test data.

The five letters, held up as one page. Identify the flip is the step a purely technical review never reaches.

Here's the turn: the 89 percent precision was real and it held steady for months. What nobody had separately tested was lead time: the flag usually arrived two to four days before a likely failure, while a technician's real schedule was booked three weeks out. Being right almost never lined up with being able to do anything about it.

Four questions a benchmark score answers nothing about.

At its worst, a compressor in an occupied building genuinely fails during a week nobody opened CircuitSense at all, and it turns out the flag had been sitting there, correct, for days.

What I would leave alone: for a low-stakes alert, like a filter due for a routine wash, the model's own accuracy really is the whole story. There is no scheduling tension to separately test.

The lesson: a model can be exactly as good as its benchmark says, and the product built on top of it can still fail completely, for a reason the benchmark was never built to catch.

Now here is the same thing as a story

The short version above is what you'd say defending a rollout in a planning review. Read this one for what it felt like the slow, quiet months before anyone thought to check.

For eight months, CircuitSense's dashboard looked healthy. Then one afternoon, Ingrid looked at a graph she hadn't opened in a while.

The third step is where a technically sound model quietly stopped reaching anyone.

Ingrid had shipped CircuitSense after months of model validation, precision, recall, calibration, all clean on held-out sensor data. Technicians opened every alert eagerly at first, curious what the new tool would catch.

One number held steady. The other one quietly went to nothing, with no single bad day to point at.

There was no complaint, no support ticket, no single bad incident. A technician would drive out on a flag, find a two-day warning window against a three-week backlog, and learn, quietly, that opening the app rarely changed anything he could actually do that week.

Over five months, the habit thinned in three quiet beats: eighteen of twenty alerts opened in month one, then about half by month three, then two of twenty by month five. Nobody flagged it, because nothing broke loudly enough to notice.

Same tool, same accuracy, and the number of people still listening had nearly disappeared.

The real question was never whether CircuitSense was 89 percent accurate. It was whether being 89 percent accurate had ever been tested against what a technician could realistically still do about it.

One full-page image to carry the whole answer: adoption did not fade. It flipped, one technician at a time.

When CircuitSense's alert format was first designed, someone said, "keep it simple, just flag the unit," and it sounded reasonable, since at the time the whole team was focused on proving the model could be right at all.

Rerun the same five months with an alert that names the lead time and the sensor reasoning behind every flag: technicians who see "four days, rising vibration on bearing two" instead of a bare warning re-engage, opens hold near 80 percent by month five, and a genuinely at-risk unit gets caught with real time to act.

What I'd tell myself, watching that graph go quiet: 89 percent was never the number that mattered. The number that mattered was two out of twenty, and it took eight months to think to look.

FLIPS, the difference that finally explained itselfNot a script for distrusting every accurate model. FLIPS is what tells you exactly which half of "it works" was never actually tested.

The recap, one line per letter: find the person is Ingrid, who owns CircuitSense's rollout, locate the habit is opening every alert on arrival, identify the flip is opening every alert versus opening none, pinpoint the old decision is the bare, unexplained flag shipped to keep the first release simple, and show the replay is an alert with lead time and reasoning that brings technicians back.

And if you want to be sure it really works, try it somewhere elseSame five letters, a fishing cooperative instead of a machine shop. The flip changes families entirely, the missing second test doesn't.

Tomasz Wibowo runs product at Saltmere Fisheries Cooperative, where StockGauge estimates fish counts from sonar and net-haul photos so biologists can set the next season's catch limits. Its technical benchmark, built on clear-water test images, hit 93 percent accuracy. Mapped onto FLIPS: find the person is Tomasz, who signs off on StockGauge's estimates. Locate the habit is biologists spot-checking roughly one haul in ten by hand, trusting the rest. Identify the flip here is a verification flip, not abandonment: once biologists noticed StockGauge's accuracy quietly worsened in murky, sediment-heavy water the benchmark never included, they stopped spot-checking and started recounting every single haul by hand, since a spot-check could no longer tell them which hauls to trust. Pinpoint the old decision is building the technical benchmark entirely from clear-water images, because that was the easiest data to collect first. Show the replay adds a turbidity reading to every estimate, so biologists spot-check only the hauls flagged as murky, instead of recounting everything.

A different flip entirely: not abandonment, but a spot-check that stopped being enough once conditions the benchmark never saw showed up.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "technical feasibility is can it be right, product feasibility is does being right change what someone does," and stop.
Cost: there's no time to test the product layer before a launch deadline. Say so honestly, and ship the model with a clearly marked "not yet tested against real schedules" label, rather than letting silence imply both questions were answered.
The model gets better, for real: if CircuitSense's lead time improves to two full weeks, that's still worth re-testing against a technician's real backlog, not assumed safe just because the accuracy number went up.

Where people run it wrong.
They treat a strong benchmark score as proof the whole product works, not just the model inside it.
They watch model accuracy after launch and never separately watch whether people are still acting on it.
They let a quiet decline in usage hide behind a healthy-looking accuracy dashboard for months.

How to use it live. The moment someone says a model is technically feasible, ask yourself: feasible to be right about what, and does being right on time change anything a real person can actually do. Name both, and the distinction explains itself.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0