What is the right fidelity for an AI prototype at the discovery stage?

How to answer this, stage by stage

This is a yes-or-no about which question to test first, not a rulebook for every prototype, so PICK does the work here.

A last note before the walkthrough ends: this pick is about order, not a ban on ever going real. Most candidates hear "low fidelity or high fidelity" and answer like it's permanent. Name the one thing only a real model can prove, and you've shown judgment instead of reciting a definition.

Let's learn

Calendra is a tool a marketing agency wants to build. It would read a client's brand guide and their last few months of posts, then write a full month of social captions and image briefs onto a calendar.

Before Calendra, a strategist at Hollis Creative built one client's month of captions and image briefs by hand. It took about 14 hours, and every caption read exactly like that client, because a person who knew the brand wrote every line.

The agency tested Calendra a different way first. A copywriter hand wrote twenty example captions, in four days, and a designer dropped them into a calendar mockup. No model touched any of it. Three client teams saw the mockup. All three loved it. One, Cortland Coffee Roasters, signed a new retainer add-on the same week, off the mockup alone.

Here is the turn. That signed retainer is not proof the tool works. It only proves people want the idea. Nobody had checked yet whether the real model could actually write a caption that sounded like Cortland, because there still wasn't a real model anywhere near the demo.

At its worst, the team skips the model check entirely, builds the real integration straight off the mockup's good reaction, and finds out only after a client has already signed, already gone live, and already started noticing the captions sound like nobody they know.

What I would leave alone. The calendar interface itself, the drag and drop, the approval screen. None of that needs to be real for a client to say yes to the idea. Fake it freely, it costs nothing and teaches plenty.

The lesson. A prototype answers one question at a time. A mockup answers "does anyone want this." Only the real model, on real examples, answers "can it actually do this." Fake the first and skip the second, and you can end up selling something you never checked exists.

What the client saw, and what was actually behind it

The mockup that sold itself

You don't need this to answer the question. Read it if you want to feel why the split has to happen on a real test, not on how good a meeting went.

Zinnia Rooksby can read a caption for four seconds and tell you whether it sounds like the client or like nobody at all. Five years running growth accounts at Hollis Creative taught her that trick, long before Calendra ever existed.

Cortland Coffee Roasters has been one of her accounts for three of those years, a small wholesale coffee brand with a voice so specific the team keeps rules for it. No exclamation marks, ever. Always call the owner Dev, never "the founder." A joke about the roaster's burnt first batch, worked into a caption at least once a month.

Hollis Creative wanted to sell a new kind of retainer: Calendra, a tool that would read a client's brand guide and their last ninety days of posts, then draft a full month of captions and image briefs onto a calendar. Every client account at once, not just Cortland. If it worked, it was the biggest thing the agency had shipped in years.

Before any of that got built, Zinnia had to decide what to actually show a client first.

She picked the cheap route. A copywriter sat down and hand wrote twenty captions for Cortland, the kind Calendra was supposed to make on its own, and a designer dropped them into a calendar mockup, four days of work. No model touched any of it. It just had to look like one had.

Three client teams saw the mockup that week, Cortland among them. All three loved it. Cortland's marketing lead signed the retainer add-on before the meeting even ended.

Leadership heard that and did the obvious thing. Skip more testing, they said, the clients already said yes, just build it. Engineering scoped the real integration, reading a live brand kit, pulling ninety days of real posts, writing drafts straight into the client's scheduling tool, at five weeks before a single real caption came out the other end.

Five weeks later, it did.

The team ran the real model against Cortland's actual brand guide and its last ninety posts for the first time. Ten sample captions came back. Nine needed a full rewrite. One used an exclamation mark. Three called Dev "the founder." None mentioned the burnt first batch, not once.

Nobody in the room had checked, before those five weeks, whether the model could even do this job. They had checked whether a client wanted it. Those turned out to be two completely different questions, and only one of them had ever gotten an answer.

The retainer went live anyway, patched captions and all. Cortland's marketing lead flagged the tone within the first week. One full billing cycle later, thirty days, Cortland canceled, not just the new add-on, the whole retainer, three years, gone.

Same product, two very different sizes of wrong

I want to say the mistake was believing the mockup. It wasn't. The mockup did exactly its job: it proved Cortland wanted a tool like this. The mistake was letting that proof stand in for a different proof it was never built to give.

So here is the choice I'd take back.

When we scoped the pilot, we skipped a two-day step: run the real model, no interface, against ten of Cortland's actual past posts, and have a strategist grade the captions before anyone signs anything. Two days. We had the time. We just didn't think we needed it, because the mockup had already gone so well.

I'd put that step back in. Fake the calendar, fake the drag and drop, fake the whole screen, none of it needs to be real for a client to say yes to the idea. But the ten captions that tell you whether the model can actually do the job, those have to come from the model, not from someone hiding behind it.

And the thing I'd tell myself, back in that scoping meeting: a client loving the idea and a model being able to do the idea are two different yeses. We only ever asked for one of them.

PICK, spelled out for a coffee client

This is a yes-or-no about which question to test first, not a rule for every prototype Hollis Creative ever builds, so PICK carries the weight here.

Try the same four letters on a newsroom pitch

Bertrand Kowalczyk edits the Mossbank Weekly Gazette, a small paper that covers one town's council meetings, school board, and little else. He wants to build Lineup, a tool that reads the week's wire feed and the council's meeting minutes, then drafts three story pitches for the paper's freelance stringers to choose from.

Before wiring Lineup to a live feed, Bertrand tests it the cheap way. He hand-picks three real past weeks of council minutes and writes, himself, the three pitches he thinks a good tool would have found. He hands the fake pitch list to two stringers and watches which one they pick.

P. Fake the pitch list first, real minutes, pitches written by a person. Keep the real model out until a separate, narrow test proves it can find a real angle in real minutes without inventing one.
I. A weak fake pitch is felt by a stringer, who shrugs and picks a different one off the list, no cost. A live model's confident wrong pitch, telling a stringer a routine budget line is a scandal, is felt by the stringer's whole week chasing a story that was never there.
C. The fake-pitch miss is cheap and visible, a stringer notices in five minutes and moves on. The live-model miss is hidden and expensive, a stringer can spend a full week reporting a pitch out before an editor catches, at copy desk, that the story doesn't exist.
K. Flip to the real feed once the model finds a real, checkable angle in at least 8 of 10 past weeks of minutes, tested blind, with an editor grading each one against what actually happened.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0