Describe the difference between a demo and a product, using a concrete example.

Veilure is a phone app. Point your camera at your own face, pick a lipstick or a foundation shade, and it renders that shade onto your skin, live, before you buy anything. Winsome Hallowell owns the try-on model surface. Domhnall Onwuachi leads the small team that built the shade-matching model underneath it. When a rival's launch teaser forced Veilure's own launch decision up by two days, Winsome had to decide what actually counted as proof the product was ready, and the evidence she reached for first was not the evidence that would have told her the truth.

How to answer this, stage by stage

Nobody is grading whether you can define "demo" and "product" like a dictionary. They're grading whether you know that a room clapping is not the same kind of evidence as a real sample, and that mixing the two up is a modeling mistake, not a marketing one.

Let's learn

A face goes in, a shade comes back rendered onto it, and it ships with nobody checking first. That last part is why the gap between the demo and the real thing mattered.

Veilure turns your own camera feed into a makeup mirror. Point it at your face, pick a shade, and it shows you what that exact lipstick or foundation would look like on you, rendered live, before you spend a rupee on it.

For the launch review, Winsome built a demo from forty photos. She and her team picked them by hand: bright, even studio light, faces looking straight at the camera, a narrow band of skin tones. Every single shade matched. The whole board watched a foundation render onto a face in under half a second and look exactly right. They approved a full launch, same afternoon.

The demo answered one question. The real launch asked a much bigger one, on a Tuesday, in a kitchen, under a warm bulb nobody chose.

Ten days after launch, Veilure pulled a sample of twelve hundred real sessions: actual users, actual phones, actual bathrooms and kitchens and car mirrors. The shade came out visibly wrong, too orange, too pale, chalky, ashy, in thirty one percent of them. For the darkest skin tones under warm indoor light, the exact pairing missing from every one of the forty demo photos, it was forty eight percent.

Here's the turn. Those extra wrong renders were never really the problem. The problem is that a demo built from forty easy photos got treated as proof the product was ready for twelve hundred hard ones, and nobody had actually checked.

What I would leave alone: the flat little color swatch shown under each shade's thumbnail in the shop doesn't need any of this. It's not a model prediction, it's the manufacturer's own listed color, shown exactly as given. Nothing about a person's face, lighting, or camera angle can make that wrong, so testing it against a thousand real sessions would test nothing real.

The lesson: a demo proves an idea can work once, for the people who built it, under conditions they picked. A product proves it works for people who didn't build it, on their own bad lighting, on an ordinary Tuesday. If the only evidence you have is the first kind, you haven't actually tested the thing that matters yet.

Now here is the same thing as a story

The short version above is what you actually say out loud. Read this one for what it cost Veilure to learn it the slow way.

Winsome Hallowell has run product for Veilure's try-on surface for two and a half years. She can spot a bad render in about half a second, mostly by watching the jawline, where a wrong foundation shade shows up first as a hard, wrong-colored line against real skin.

Domhnall Onwuachi leads the small team underneath her that built the model doing the actual matching: reading a face from camera pixels and deciding which of Veilure's four hundred shades belongs there.

The Tuesday before the board meeting was supposed to be a quiet one. Winsome spent the morning doing the thing she was good at: going through a folder of hundreds of internal test photos and pulling the best forty, the ones where the render landed clean every time. By lunch she had a demo that made a lipstick look correct on every single face in it. She was proud of it. She had earned being proud of it.

Then, that same morning, a rival app dropped a teaser for a near-identical live try-on feature, slicker marketing, a countdown clock on the landing page. Leadership panicked a little. The board meeting that was supposed to happen Friday got moved to that same afternoon, with one line attached: if the demo's ready, we go the same day.

Two hours between the teaser and the meeting. Two things sitting on Winsome's desk. She only carried one of them into the room.

Two things were sitting on Winsome's desk that afternoon. The forty-photo demo, finished and beautiful. And Domhnall's twelve-hundred-session eval, the representative one, maybe sixty percent run, no clean number yet, definitely not ready to present. She didn't have time to wait for the second one. She walked into the meeting with the first.

The board watched every one of the forty photos land correctly. Winsome herself got swept up in how good it looked, the way you do when a thing you built goes exactly right in front of people whose opinion matters. She recommended launching that day. The room agreed in about four minutes.

By the following Thursday, eleven days after launch, the app store reviews had started using the words grey and ashy. A beauty writer with a decent following posted a screenshot of her own render next to Veilure's own marketing image, side by side, captioned "guess which one is the ad." Refund requests came in faster than new sign-ups for most of that week.

I want to say the problem is that the model got things wrong. It did get things wrong. But that's not really the story. Nobody at Veilure had a number in their head for "ready." They had a feeling, and the feeling only had two settings: the room looked convinced, or it didn't. Forty good photos flipped it to convinced. Nothing in that room was built to notice that the forty photos and the twelve hundred real ones were answering two completely different questions.

Months earlier, when Domhnall's team first proposed the representative eval set as a standing gate before every launch decision, it got scoped down. Building it properly meant real photography across nine skin-tone bands, four lighting setups, and three head angles, real people, real time, and nobody wanted to hold up a fast-moving roadmap for it. So it became a thing the team ran when there was time, not something a launch decision actually had to clear first.

Everything that made the room feel certain sat in the same dangerous corner: hand-picked, and convincing because of it.

Here's the replay. Say the board meeting again, but this time Winsome walks in with both numbers instead of one. The forty-photo demo, spotless. And the real eval, seventy four percent complete, current read: nineteen percent mismatch overall, climbing to forty four percent in the hardest cell. She recommends a two-week hold instead of a launch.

Domhnall's team spends those two weeks doing exactly what the real numbers point at, mostly recalibrating how the model reads warm indoor light against darker skin. They don't touch the render engine itself, which never had a problem. Mismatch rate on the full representative set: nineteen percent at week zero, eleven at week one, four at week two, under the five percent line the team had agreed on beforehand. Veilure launches in week three instead of that afternoon, to a market that never sees a bad render make the news.

One version of this story ships on the strength of forty photos and spends its first two weeks fighting reviews. The other spends two weeks fixing a known number before a single stranger sees it. Same model, same team, same two weeks either way. The only thing that changed was which number got to be the evidence.

What I'd tell myself, watching Winsome carry those forty photos into that meeting: a demo that makes a room go quiet is real evidence of something. It's just not evidence of the thing everyone in the room decided it was evidence of.

PICK: what forty perfect photos can't prove

Not a rule about being cautious. PICK only earns its keep here if it makes you name, out loud, which of the two mistakes actually costs more, and it isn't the one that feels riskier in the room.

One mistake shows up the same week and costs a little pride. The other hides behind a good-looking number until strangers find it for you.

Four branches, one question asked before anything ships: did the evidence cover the real spread, and did it check the worst group, not just the average.

The trade worth saying out loud: building and running the representative eval before a launch decision costs real calendar time, in this case about two to three weeks of real photography and labeling across skin tone, lighting, and angle, time a forty-photo demo simply doesn't need. That's slower and more expensive up front. It's worth paying, because the alternative cost, a public launch that fails on exactly the users it needed to work for, is bigger every single time.

And if you want to be sure it really works, try it somewhere else

Same four letters, an HVAC diagnostics tool instead of a makeup mirror, and the fix a field team almost trusted would have been wrong for just as specific a reason.

Coilwise runs a phone app for HVAC field technicians. Point it at a rooftop unit, and it tells you which part is likely failing, a bad capacitor, a clogged coil, a dying motor, before the technician opens the panel. Its demo, built from photos of a handful of clean, well-lit units on a training rooftop, diagnosed every fault correctly. Real units live in dim mechanical rooms, coated in dust, lit by a phone's own flash, across a dozen brands the demo never touched.

Coilwise's PM had the same two options Winsome did. One tests the model against the job it will actually be asked to do. The other tests it against the job that was easiest to photograph.

Position: a demo unit, clean and well lit, proves the model can read a fault when nothing is fighting it. A product has to read the fault through dust, glare, and a brand the model has barely seen. Impact: launching on the demo alone risks technicians trusting a wrong diagnosis and swapping the wrong part, which costs a customer a second visit and Coilwise a technician's trust in the tool. Cost asymmetry: a slower rollout, tested first against a real sample of dim, dirty, mixed-brand units, costs a few weeks of field data collection. A wrong diagnosis in the field costs a wasted service call and a technician who stops trusting the app on every job after, not just the ones it got wrong. Kill criteria: does the model hold up against a large, representative sample of real field photos across brand, lighting, and unit condition, not just the demo units it was shown on a clean rooftop.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: name the kill test before anything else gets discussed.
Cost: no budget for a full representative sample before the deadline. Fine, but test the highest-risk cells first, the darkest skin tones or the dirtiest mechanical rooms, not a bigger pile of the same easy inputs.
The model got better, for real: say a new checkpoint scores near-perfect even on the hard cells. Still don't skip the representative retest. "Better on average" can still hide one bad cell nobody looked at.

Where people run it wrong.
They mistake a room going quiet for the product actually working, when it only proves the forty photos were chosen well.
They test more of the same easy inputs to make the number look better, instead of harder new ones to learn something true.
They patch visible failures one ticket at a time as users report them, instead of asking whether a representative test was ever run at all.

How to use it live. If you're ever handed a glowing demo mid-interview and asked whether you'd ship it, buy yourself a second by asking one thing out loud: "how was that test set actually chosen." That question is the whole method, said as a question instead of a rule.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0