CaseIntermediateAI Opportunity & Model Strategy / Build vs buy vs fine-tune decisions / #14
Compare time-to-market for the three paths on a feature you specify yourself.
ORDERthe option that sounded fastest was stuck in procurement while the cheapest one already had an answer
FreshFleet Grocery runs same-day grocery delivery from regional warehouses. RipenessCheck reads a photo a picker takes of a piece of produce and decides ship, hold, or discard. Tamsin Okwuosa is the AI PM who has to pick a path for it, and Renke Solvang is the engineer whose weekend prototype forced the comparison to actually happen.
The direct answer
Building with a well-crafted prompt around a general vision model reaches a testable version fastest, in about two weeks, because there's no data collection or procurement to wait on. Buying looks like it should be faster since the technology already exists, but a vendor contract and security review usually add three to four weeks before a single photo gets processed. Fine-tuning is slowest by far, eleven weeks or more, because collecting and labeling real produce photos dominates the timeline, not the training itself.
Do this, in order
Build a real eval set before comparing any of the three paths.Why: without one, "time to market" gets measured on vibes instead of a real, shared bar.
Test the prompt-based build path first, it's the fastest to a real answer.Why: it needs no procurement and no data collection, just an afternoon of prompt work.
Start the buy path's procurement in parallel, not instead of building.Why: contract review and security sign-off run on their own clock, regardless of how ready the technology is.
Treat fine-tuning as the slow path from day one, and plan around it.Why: real data collection, not training time, is what makes it take three months instead of three weeks.
Never let an ungoverned prototype spread without the same eval set behind it.Why: a fast build with no measured accuracy is a guess wearing a shipped feature's clothes.
Re-rank the three paths once real numbers exist, not before.Why: "buy should be fastest" was a reasonable guess. It was also wrong, and only real numbers caught that.
How to answer this, stage by stage
Nobody is scoring whether you picked the objectively fastest path. They're scoring whether you tested the assumption instead of trusting it.
Stage 1
Scope it to one real feature you specify yourself
Say it like this
"I'll pick a real one. FreshFleet Grocery does same-day delivery. RipenessCheck reads a photo a picker takes and decides ship, hold, or discard. Tamsin owns the build-buy-finetune call, and I'll compare real time-to-market across all three."
Why this works
Shows you can specify a concrete feature instead of waiting to be handed one, which is exactly what this question is testing.
Stage 2
Say your structure out loud before any content
Say it like this
"I'll run this as ORDER. Outcome, what all three paths are actually racing to hit. Reversibility, which is cheapest to undo. Dependency, what has to happen before comparing them means anything. Evidence, what gets tested cheaply first. Rank, the real order, timed."
Why this works
Signals a repeatable comparison method instead of a guess about which path "should" be fastest.
Stage 3
Reframe the question: "buy is fastest" is an assumption, not a fact
Say it like this
"Most people assume buying is the fast option because the technology already exists somewhere. That assumption skips the part where a vendor contract, a security review, and a procurement queue all run on their own clock, no matter how ready the product is."
Why this works
This is where the answer separates from a surface-level "buy is usually faster" cliche.
Stage 4
Give the one comparison: the real timeline for all three
Say it like this
"Here's how it actually ran. Building with a prompt around a general vision model was testable in two weeks. Buying a vendor's tool took three and a half weeks, mostly procurement and security review. Fine-tuning our own model took eleven weeks, almost all of it collecting and labeling real produce photos."
Why this works
This is the direct answer, stated as three real numbers instead of a ranked guess.
Stage 5
Prove it with the compressed evidence
Say it like this
"On a 300-photo eval set, the build path hit 81 percent correct ship-hold-discard decisions after two weeks. The vendor's out-of-box demo hit 76 percent after three and a half weeks, tuned for a different warehouse's lighting. Fine-tuning on our own 5,000 collected photos reached 93 percent, but only after eleven weeks."
Why this works
Gives the interviewer real numbers for both speed and accuracy, so the comparison isn't just about which one shipped first.
Stage 6
Name the AI-specific reasoning and the trade-off being accepted
Say it like this
"The honest reason fine-tuning is slow isn't the training run, it's that a model needs real, labeled produce photos across every ripeness stage and lighting condition, and that data doesn't exist until someone photographs and labels it. We accepted a lower accuracy ceiling from the build path in exchange for shipping in two weeks instead of eleven, and planned to swap in fine-tuning later once the data existed anyway."
Why this works
This is the load-bearing judgment. It only makes sense because the model's accuracy is bound by real training data, not by generic engineering effort.
Stage 7
Say what changes the ranking, then close on one line
Say it like this
"If FreshFleet already had a produce photo archive sitting around, fine-tuning's eleven weeks would mostly collapse, and the ranking would shift. As it stood, build first, buy in parallel as a backup, fine-tune once the data existed, in that order, timed."
Why this works
Closes with the condition that would reorder the ranking, and restates the direct answer in one breath.
Let's learn
RipenessCheck is meant to replace a picker's unaided visual judgment with a photo-based check, so a warehouse worker gets a clear ship, hold, or discard call on every piece of produce instead of guessing.
Nothing about the three paths could be honestly compared until one thing existed first: a real eval set.
Before that eval set existed, the plan was simple on paper: request vendor quotes for an existing produce-inspection product, since buying an already-built tool sounded like the obvious fast option, and revisit building only if the vendor fell through.
Time to a testable version, by path
BuildBuyFine-tune
The path everyone assumed was fastest, buying, landed in the middle. Procurement has its own clock, and it doesn't care how ready the product is.
Comparing three paths on "time to market" only means something once there's one fixed answer to check each path against.
Knowledge spark: why does buying take longer than the technology itself suggests?
A vendor's product might genuinely be ready to use today. What isn't ready is your own company's contract review, data-sharing agreement, and security sign-off, none of which move faster just because the underlying technology is mature. Time-to-market for "buy" is really time-to-signed-contract, not time-to-working-demo.
The fastest path was never the one that already existed somewhere else. It was the one with nothing to wait for.
Here's the turn: once real numbers existed, buying wasn't just slower than expected, it also scored worse. The vendor's product had been tuned for a different warehouse chain's lighting and produce mix, and it underperformed FreshFleet's own two-week prompt-based build on the exact same eval set.
Accuracy achieved, plotted against real weeks invested
Actual path: build, then fine-tuneIf FreshFleet had waited on buy alone
Waiting for the "fast" option would have delivered a worse number, later, than just building started right away.
The choice I would take back
Letting Renke's weekend build prototype spread to real picker shifts before it had the same eval set the vendor comparison would eventually use. That made sense in the moment, since it was clearly faster than the paper process it replaced. It stopped making sense once nobody could say how accurate it actually was, until a near miss forced the question.
What I would leave alone: FreshFleet's produce weight verification at checkout, a plain scale reading against an expected range, doesn't need this comparison at all. There's no judgment call being made, no path to fine-tune.
The lesson: "buy is fastest" is a guess about the vendor's product, not about your own company's procurement queue. Test all three paths against the same real eval set before ranking them, because the ranking that seems obvious on paper is often backwards.
Now here is the same thing as a story
The short version above is what you'd say out loud in the room. Read this one for what it actually felt like watching an assumption about "the fast option" turn out to be wrong.
Tamsin Okwuosa had sent the vendor request for a produce-inspection tool in the first week of the quarter, expecting a quick pilot agreement. Renke Solvang, an engineer on the team, spent a slow weekend building a rough prompt-based version around a general vision model, mostly out of curiosity about how far a simple approach could get.
By week two, pickers on the late shift had quietly started using Renke's prototype, because it was faster than flagging a supervisor for every uncertain piece of produce. It wasn't an official rollout. Nobody had measured its accuracy against anything.
The prototype had already been running for two weeks before anyone put a real number next to it.
In week three, a supervisor doing a routine random spot check found a pallet of bruised avocados that the prototype had approved to ship. Nobody downstream had caught it yet. It never reached a customer. But it was close enough that Tamsin pulled the prototype from picker shifts that same afternoon and asked for a real comparison before anything went back into use.
The near miss wasn't really about the prototype's accuracy. It was about shipping something nobody had measured yet.
We weren't comparing three technologies. We were comparing three different clocks, and only one of them had actually started.
Tamsin's team built the 300-photo eval set that week, with two warehouse leads independently agreeing on a ship-hold-discard label for each photo. Tested against it, Renke's build hit 81 percent, already better than most people expected from a weekend project.
The vendor's product wasn't just slower to arrive. Once it arrived, it scored worse than the two-week build it was supposed to beat.
Tamsin never had a fixed rule for exactly when "buy" stopped being worth waiting on. It came down to a feeling with two settings: either the vendor's head start on the technology was worth the procurement delay, or a fast, honestly-measured build could close that gap before the contract even finished review. The vendor's demo landing at 76 percent, two weeks after the build's 81, settled which setting this was.
Whatever path won, the actual decision it had to produce was always this same, small set of parts.
Back when the vendor request went out first, assuming "buy" would be the fast option wasn't an unreasonable call, the underlying technology genuinely did already exist. It stopped being reasonable the moment procurement's own timeline turned out to run longer than building something from scratch.
Here's the replay: build shipped in two weeks at 81 percent, with pickers now getting a measured, eval-tested tool instead of an unmeasured prototype. The vendor path was formally declined at week four once its 76 percent came in below the build's number. Fine-tuning on FreshFleet's own growing photo archive finished at week eleven, reaching 93 percent, and replaced the build path once it cleared the bar.
One version of this story waits on a vendor contract for a month, assumes the wait was worth it, and never notices the number that came out the other side was worse than what a weekend prototype already had. The other tests all three against the same real eval set and lets the numbers, not the assumption, decide the order.
What I'd tell myself, standing in that spot-check meeting: "buy is probably fastest" is a sentence about the technology, not about your own company's procurement queue. Test the assumption before the vendor's technology gets credit it hasn't actually earned yet.
ORDER, run on an assumption that turned out backwardsNot a script for always building instead of buying. ORDER is what stops "obviously faster" from going untested.
O
Outcome. What are all three paths actually racing to hit?
A RipenessCheck call a picker can trust, measured against a real 300-photo eval set, not just "whichever ships first." Speed only counts once accuracy clears a usable bar.
Without naming this, "fastest" and "best" get treated as the same question, and they aren't.
R
Reversibility. Which path is cheapest to walk back if it's wrong?
Build: swap a prompt in an afternoon. Buy: a signed vendor contract, harder to unwind once integrated. Fine-tune: real weeks of committed data collection and training, the hardest to reverse.
This is why build gets tried first, not because it's automatically the best, but because being wrong about it costs almost nothing.
D
Dependency. What has to be true before comparing the three even means anything?
A real, agreed eval set. Without it, "buy felt slower" or "build seemed fine" are just impressions, not a comparison anyone could defend.
This is the step the ungoverned prototype skipped, and it's why nobody could catch the near miss until a supervisor got lucky.
E
Evidence. What could get tested cheaply before committing to any one path?
The build path, tested in two weeks at zero procurement cost. The vendor's demo, tested against the same eval set once access was available. Both ran before fine-tuning's real data-collection cost was fully committed.
Cheap evidence, gathered in parallel, is what caught the vendor's underperformance before a full year's contract got signed.
R
Rank. State the real order, timed.
Build shipped week 2 at 81 percent. Buy was declined at week 4 after scoring 76 percent. Fine-tune finished week 11 at 93 percent and replaced build once it cleared the bar.
This is the direct answer, the order that actually happened, not the order everyone assumed on paper.
The recap, one line per letter: outcome is a trustworthy ship-hold-discard call measured against a real eval set, reversibility is why build gets tried first and fine-tune last, dependency is the eval set nothing else can be honestly compared without, evidence is testing build and buy in parallel before fine-tuning's cost gets committed, and rank is the real, timed order: build, then buy declined, then fine-tune.
And if you want to be sure it really works, try it somewhere elseSame five letters, a home-security company instead of a grocery chain. This time buy actually does win, once the numbers come in.
Marlowe Quintrell is the AI PM at Vantage Doorstep, which sells video doorbells, and specifies PorchWatch, a feature that tells a homeowner whether a package left on the porch is likely to still be there in an hour based on foot traffic patterns in the frame. Mapped onto ORDER: outcome is a package-risk alert accurate enough that homeowners trust it instead of muting it. Reversibility ranks the same way, build first, buy in parallel, fine-tune last. Dependency is the same kind of real eval set, a few hundred real porch clips with an agreed risk label. Evidence comes back differently here: a two-week build reached only 68 percent, genuinely too low, while a specialist vendor already selling exactly this kind of foot-traffic model matched 89 percent in three weeks, once procurement moved fast because Vantage already had a standing security-vendor relationship. Rank ends one step earlier than FreshFleet's did: buy, because the vendor's existing model, trained across thousands of doorbell cameras, already covered a harder problem than a from-scratch prompt could close in weeks, and fine-tuning never got a serious look once buy cleared the bar.
Same dependency order as FreshFleet's. This time the vendor's real head start actually held up once tested.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Skip straight to "test build and buy in parallel against the same real eval set, don't assume which one's faster," and stop.
Cost: no time to build a full eval set before a decision is due. Say so honestly, and commit to a smaller, twenty-photo version as the very next step, not a guess at which path to skip to.
The model got better, for real: say a future general model handles ripeness detection at 90 percent out of the box, no fine-tuning needed. Rerun the same comparison anyway, a better model just means build wins even more clearly, it doesn't change which path gets tested first.
Where people run it wrong.
They assume "buy" is fastest because the technology already exists, without accounting for their own company's procurement and security-review timeline.
They let a fast, unmeasured prototype spread into real use before it has the same eval set any other path would be judged against.
They treat time-to-market as the only number that matters, and skip checking whether the fastest path also clears a usable accuracy bar.
How to use it live. The moment an interviewer asks you to compare time-to-market across build, buy, and fine-tune, don't reach for the one that "should" be fastest. Ask yourself: what's actually blocking each path today, a missing prompt, a missing signature, or missing data? That question alone usually reorders the ranking on the spot.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits comparing time-to-market across build, buy, and fine-tune?
Tap to flip
ANSWER
ORDER: outcome, reversibility, dependency, evidence, rank. It ranks the three paths by what's actually blocking each one, not by which one sounds fastest on paper.
2 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Tamsin Okwuosa, the AI PM who owns RipenessCheck's build-buy-finetune decision at FreshFleet Grocery. Renke Solvang is the engineer whose weekend prototype forces the real comparison.
3 · THE ORDER
What's the actual ranked order that played out, timed?
Tap to flip
ANSWER
Build shipped at week 2, 81 percent. Buy was declined at week 4, 76 percent. Fine-tune finished at week 11, 93 percent, and replaced build once it cleared the bar.
4 · THE DEPENDENCY
What had to exist before any of the three paths could be honestly compared?
Tap to flip
ANSWER
A real, agreed eval set, 300 photos with a ship-hold-discard label two warehouse leads agreed on independently. Without it, every comparison would have just been opinion.
5 · THE OLD DECISION
What old decision does this answer take back?
Tap to flip
ANSWER
Letting the weekend build prototype spread into real picker shifts before it had a real eval set behind it. Reasonable in the moment, since it beat the old paper process. Wrong once nobody could say how accurate it actually was.
6 · THE NUMBER
Fill in the blank: the build path reached ___ percent accuracy in 2 weeks. The vendor's demo reached ___ percent, in 3.5 weeks.
Tap to flip
ANSWER
81 percent, then 76 percent. The path everyone assumed was ahead on quality actually landed behind, once both were tested against the same eval set.
7 · THE REPLAY
Same near miss, with the real eval set built before the prototype ever reached picker shifts. What changes?
Tap to flip
ANSWER
The 81 percent accuracy gets measured in week 2, before any pallet ships, and the near miss never happens because the tool's real limits are known from the start instead of discovered by a lucky spot check.
8 · CROSS PRODUCT TRANSFER
Section 4 runs ORDER again on a different product. Which one, and how does the outcome differ?
Tap to flip
ANSWER
Vantage Doorstep's PorchWatch. Unlike RipenessCheck, buy actually wins here, the vendor's model, trained across thousands of doorbell cameras, beat a from-scratch build the moment real numbers came in.
Check yourself Score: 0 / 0
Fill in the blank
1. Fill in the blank: building with a prompt reached a testable version in ___ weeks, buying took ___ weeks, and fine-tuning took ___ weeks.
Show hint
Look at the "Time to a testable version" bar chart.
Show answer
2 weeks, 3.5 weeks, 11 weeks. The path everyone assumed was fastest, buying, actually landed in the middle once procurement's real timeline was counted.
Multiple choice
2. Why did buying take longer than the underlying technology alone would suggest?
A. The vendor's product wasn't actually finished yet.
B. Contract review and security sign-off run on their own timeline, regardless of how ready the product is.
C. FreshFleet's warehouses didn't have the hardware to run the vendor's tool.
D. The vendor required a custom fine-tuned version before delivery.
Show hint
Look at the knowledge spark on why buying takes longer than expected.
Show answer
B. "Time to market" for buying is really time to a signed contract, and that clock runs independently of how mature the underlying technology already is.
True or false
3. True or false: once tested, the vendor's product scored higher accuracy than FreshFleet's own two-week build.
True
False
Show hint
Look at the accuracy-versus-weeks line chart.
Show answer
False. The vendor's demo scored 76 percent, lower than the build path's 81 percent, because it had been tuned for a different warehouse's lighting and produce mix.
Short answer, where it wouldn't matter
4. Name a feature at FreshFleet where this whole three-way comparison would NOT need to be run in full, and say why not.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: Produce weight verification at checkout, a plain scale reading against an expected range. There's no judgment call involved and nothing to fine-tune, so the comparison doesn't apply.
Short answer, apply it yourself
5. Think of a build-versus-buy decision you've seen made on assumption rather than evidence. What would a two-week test of the "obviously faster" option have actually shown?
Show hint
Think about a case where the "obvious" choice skipped an internal process, like procurement, legal review, or security sign-off.
Show answer
Model answer: A company assumed buying an off-the-shelf chatbot vendor would be faster than an internal build, but a security review of the vendor's data-handling terms alone took six weeks, longer than a scoped internal prototype would have taken to test.
Short answer, work the number
6. If the vendor's accuracy had come in at 85 percent instead of 76, would declining the buy path at week 4 still have made sense?
Show hint
Compare 85 percent against the build path's 81 percent and the fine-tune path's eventual 93 percent.
Show answer
Model answer: Less clearly. At 85 percent, the vendor would have beaten the two-week build, and the honest move would be to weigh that quality gain against 1.5 extra weeks of procurement, rather than declining it outright.
Before you close the answer
Why this works
Tests whether you'll test an assumption about which path is fastest instead of trusting it, and whether you know that "time to market" has to include your own company's process, not just the vendor's readiness.
Follow-up traps
"Isn't testing all three paths in parallel just more work than picking one?" Response: testing build and buy together took two extra weeks of vendor demo access, far cheaper than discovering four weeks into a signed contract that the product underperforms a prototype that already existed.
"Couldn't the near miss have been avoided by just being more careful?" Response: the real fix wasn't more caution from the pickers, it was never letting an unmeasured tool reach a real shift in the first place, which is a process decision, not a vigilance one.
If pressed
The fine-tuned model that eventually replaced the build path still runs the same picker-override step underneath it. A low-confidence call still routes to a person, so the 93 percent accuracy number was never treated as a reason to remove human judgment entirely.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
