Describe how you would prototype a multimodal feature.

How to answer this, stage by stage

Nobody is grading whether you land on exactly twenty-eight. They're grading whether you name the real input combinations before touching a number, whether the riskiest ones get more than an equal share, and whether the total survives a check against how long staging it actually takes. Eight moves get you there.

Let's learn

The feature is a camera button and a microphone icon, sitting on the same screen inside a hardware retailer's app. Point your phone at whatever's broken. Hold the mic button and describe it. The assistant tells you which aisle to visit, or tells you to put the phone down and call a professional instead.

Before asking anyone to build it, someone has to prove the assistant can actually read a photo and a voice line together, the way a real customer hands them over. Not the way the person who built it hands them over.

The first way to prove it: pick six common repair jobs, take one clean photo of each, good light, camera held steady, and record one short, clear voice line describing exactly what's wrong. Walk each one through the logic by hand. Six for six. Fast. Clean. Convincing in a room.

The second way: name every real combination of photo and voice a customer could actually hand over, including the messy ones, and test a real spread of repair problems inside each one.

At its worst, that gap costs more than a wrong aisle. It's the case where the photo shows a pipe and the voice line mentions a smell, and the assistant answers the photo instead of listening to both.

The choice I would take back. The first test set was six photo-and-voice pairs, all recorded by the product manager herself, in her own kitchen, good light, calm voice, one input per repair category. The fix: five real input combinations (clear photo plus short voice, blurry photo plus long voice, photo only, voice only, and photo and voice that disagree), a floor of four repair cases each, twenty. Double the floor for voice-only and mismatched signals, since those are the two places the assistant can miss a hazard it never actually saw, adding eight more. Twenty-eight test cases, not six, before the feature goes anywhere near a real store.

What I would leave alone. The clear-photo and blurry-photo combinations don't need the extra floor. Guess a dead light switch wrong and a customer just walks to the wrong aisle once, thirty seconds lost, nobody hurt. Not every combination is worth doubling, only the ones where a miss costs something a customer can actually feel.

The lesson. Six for six sounds like proof. It's only proof of whatever the person building the test was careful enough to record cleanly.

Now here is the same thing as a story

Skip this if you already believe a self-recorded demo and a real customer's input aren't the same test. Read on if you want to feel why they aren't.

For three weeks, the best number in Ngozi Achike's whole pitch deck was six. Six for six.

Ngozi runs product for the "what's wrong with this" feature at Anvilcross Hardware, a retailer with about forty stores and an app most of its regulars keep on their home screen. The idea was simple to say out loud: point your phone at whatever's broken, hold the mic button, describe it, and the app tells you the aisle, or tells you to call someone instead of trying it yourself.

Before pitching anyone, Ngozi wanted proof the idea actually worked, not just a slide. So one afternoon she picked six of the store's most common repair jobs, a leaky faucet, a stuck lock, a cracked tile, a dead switch, a jammed drawer, a wobbly deck board, and for each one she took a clean photo in her own kitchen or garage, good light, camera held steady, and recorded a short, clear voice line describing exactly what was wrong. Six inputs. Six correct answers. Right aisle, right fix, every time.

She put it in front of the executive team that Friday. Six for six, camera and voice working together, ready to build. She asked for a small pilot in three stores. She got it.

Two weeks in, an associate on the transcript-review team flagged one clip for her. A customer, standing near a water heater, had said something like this into the mic: "There's a hissing sound and kind of a rotten egg smell, been going on since this morning." But the photo never actually got the water heater in frame. It mostly showed the wall and a length of pipe behind it.

The same camera-and-mic screen, fed two different ways. One only ever saw the input its own product manager knew how to record. The other saw whatever a real customer's hands and voice actually gave it.

The assistant read "hissing" against a pipe shape in the photo, called it a loose fitting, and told the customer to tighten the connection themselves.

That's a gas smell near a water heater. That is not a tighten it yourself answer. It's a leave the house and call the gas company answer.

Nobody got hurt. The associate on the transcript-review team caught the phrase "rotten egg smell" and called the customer directly before they touched anything. But Ngozi couldn't stop turning it over. None of her six demo inputs had ever included a case where the photo missed the actual hazard. None of them had a voice line that mentioned something the camera never saw.

Here's the part that actually cost something. It wasn't the one bad transcript. It was that three stores had already been running the pilot for two weeks on an assistant that had never once been asked to handle a photo and a voice line that didn't agree. Finding that out from a real customer's near miss, instead of from a test case built weeks earlier, meant the near miss was doing the job a test case should have done.

So here's what I'd take back. Building that first test set from one clean photo and one clear voice line per repair category, recorded by the person who already knew the right answer, instead of sizing it by every real combination a customer would actually hand over.

Ngozi rebuilt the test set properly: five input combinations, a floor of four repair cases each, twenty, doubled to eight for voice-only and mismatched signals, twenty-eight total, and this time nobody recorded the inputs themselves. Every case was written to match a pattern the store's own repair logs already showed happened. Run against the assistant as it stood, twenty of the twenty-eight came out right on the first pass, and every miss sat in the same two places: voice-only cases, and the ones where the photo and the voice disagreed. Two fixes: flag any voice line that mentions a smell or a hissing sound for a "call a professional" answer regardless of what the photo shows, and escalate any case where the photo and the voice line don't obviously match. Rerun: twenty-six of twenty-eight.

The real feature launched chain-wide that quarter. In its first month, it gave the right aisle or the right "call a professional" flag on 89 percent of cases on the first try. The six-clean-input version, run against that same real spread, would have handled an estimated 61 percent of it blind. And the case that used to slip past a photo that never quite caught the hazard, a smell or a sound mentioned only in the voice line, gets flagged for a professional now, on the first pass.

The thing I'd tell myself, the Friday I called six for six proof: a test built from six clean inputs you staged yourself will always look finished. That's exactly what makes it dangerous the moment a real customer hands you something you never tried to build wrong.

BOUND, sized for a camera and a microphone

This is a sizing question about how many real input combinations to test and how many repair cases inside each one, not a person's trust flipping between two settings, so BOUND fits and FLIPS doesn't.

B, break it down. The number of test cases equals the number of real photo-plus-voice input combinations the feature will meet, times a floor of repair problems to test inside each one. Not one clean photo and one clear sentence per repair type.
O, own the numbers. Five real combinations: clear photo plus short voice, blurry photo plus long voice, photo only, voice only, and photo and voice that disagree. A floor of four repair cases each (faucet leak, stuck lock, cracked tile, dead switch) gives five times four, twenty. Voice-only and mismatched-signal combinations carry real risk if the assistant guesses wrong, so their floor doubles to eight each, adding eight more. Total: twenty-eight.
U, use a range. If the prototype only tests the diagnosis call itself, three cases per combination is probably enough, and the total drops to about fifteen. If the real feature chains a few more steps, an aisle lookup, a stock check, a "this needs a licensed pro" escalation, each one is a place the workflow can branch, and covering those branches pushes the floor toward seven cases per combination, toward fifty. Start at twenty-eight, the plain four-case floor, and only widen it once the real fallback logic is decided.
N, nail the sanity check. Twenty-eight test cases, staged for real, finding or faking the problem, taking the photo, recording the voice line, writing down the right answer, at about twenty minutes each, comes to a little over nine hours. More than a working day, which is the whole point of a prototype check: cheap enough to run before the real feature gets built. If that number ever crept past what it'd take to actually build the real aisle-lookup logic, staging would have quietly turned into doing the job by hand instead of testing it.
D, direction. The case floor moves this more than the combination count does. Raising the floor by one case across all five combinations, with the two doubled ones counting double, adds seven. A whole new combination only adds four. If this number needs to shrink, the floor is the first place to look, not the combination list, since cutting a combination means an entire kind of customer input goes untested.

And if you want to be sure it really works, try it somewhere else

A plant-care app runs the same idea on a leaf instead of a leaky pipe. A customer photographs a struggling houseplant and describes what's been happening out loud: how often they water it, whether it sits near a sunny window, whether they just repotted it.

B, break it down. The number of test cases equals the number of photo-plus-voice input combinations, times a floor of plant problems to test inside each one. Four named combinations: clear photo plus short voice, blurry or backlit photo, photo only, and voice only.
O, own the numbers. A floor of three plant-problem cases each, yellowing leaves, brown spots, wilting despite regular watering. Four times three is twelve. Photo-only gets that floor doubled to six, since without hearing about watering habits or a recent repot, overwatering and a pest problem can look nearly identical in a photo alone, adding three more. Total: fifteen.
U, use a range. Testing only the diagnosis call itself, two cases per combination is close to enough, dropping the total to about eight. Chaining in a plant-database lookup and a nursery-referral flag for anything that looks like a pest infestation pushes the floor to five cases per combination, toward twenty.
N, nail the sanity check. Fifteen test cases, staging a real struggling plant or borrowing one, taking the photo, recording the voice note, at about fifteen minutes each, comes to a little under four hours. Well under half a day, reasonable for a feature with lower stakes than a gas smell.
D, direction. Same shape as Anvilcross's. Raising the floor by one case across four combinations, with the doubled one counting twice, adds five. A new ordinary combination only adds three. The floor still moves it more.

Swap the trigger and it still runs.
Speed: an interviewer caps the answer at 90 seconds. Skip straight to the split: floor of 4, doubled to 8 for the two riskiest combinations, 28 total, about nine hours to stage. The build-up backs it up if they ask.
Cost: instead of a case count, a manager caps the prototype at one day of staging time. Work backward: at 20 minutes a case, that's about 24 cases in eight hours, cover the two doubled combinations first and leave the rest for the next pass.
The model got better: a newer vision model rarely misreads a well-lit photo anymore. The floor doesn't drop on its own. Rerun the voice-only and mismatched-signal slices specifically, since a clean overall pass rate hides the one combination that used to fail.

Where people run it wrong.
They size the prototype by how convincing six clean, self-recorded inputs look, instead of a floor times a combination count.
They record every test input themselves instead of staging a real, unpicked spread, which quietly leaves out exactly the input a real customer would hand over.
They let the case count grow until staging it costs more hours than building the real feature would, so the prototype stops paying for itself.

How to use it live. Say the equation before any number: "the test case count is combination count times a per-combination floor, doubled wherever the model can't see the hazard directly or the signals disagree." That buys the time to name real combinations instead of guessing a round number that sounds thorough.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0