CaseAdvancedShipping & Model Lifecycle / Pilot design and POC-to-production / #17

How do you price a pilot, and should you charge at all?

The direct answer
Charge something for every pilot, even a flat fee that is small next to the eventual contract, instead of running it free. A fee is the cheapest test of whether a customer will actually use the tool. The one time I would not charge: when this specific customer's own work sits in a project type my eval set barely has, badly enough that the data is worth more than the fee.
Do this, in order
  1. Charge a flat fee for every pilot, instead of running any of them free.Why: a price is the cheapest filter for whether a prospect will actually run real work through the tool, not just take a free look.
  2. Waive the fee only when the prospect's own work fills a real hole in your golden set.Why: that data is worth more than the fee, and losing that one customer over price leaves a blind spot you will not find until a live job goes out wrong.
  3. Set the fee low enough that it rarely costs you a customer you actually need.Why: the fee is meant to filter out tourists, not price out the exact customers whose hard cases you are trying to buy.
  4. Measure a pilot by real work run through the tool, not by attendance on a kickoff call.Why: an uncommitted customer looks engaged on a call and still hands you nothing usable.
  5. Keep a guardrail in front of every unreviewed number the tool proposes, paid pilot or not.Why: a paying customer still deserves protection from a confident, wrong estimate reaching a real bid.
  6. Revisit the fee once you have enough pilots to know your real no-show rate.Why: the flat number is a starting guess, not a rule fixed at launch.

How to answer this, stage by stage

this is a should-you-charge tradeoff, not a range of options, so PICK carries it. commit to a position, then earn it.

1
Scope it to one concrete pilot
Say it like this
"Let's make this real. Say I run pilots for Bidframe at Trussgate, a tool that reads a project's plans and spec sheets and recommends a starting price for every line item. A general contractor wants six weeks with it before they'll sign anything. I need to decide whether that six weeks costs them money."
Why this works
Stops the answer from floating at "it depends on the deal" and gives the interviewer one real decision to push on.
2
Say your structure out loud
Say it like this
"I'll take a position first, then say who eats each kind of mistake and in what units, then name which mistake actually costs more, then say what evidence would flip my answer. That's PICK, in that order."
Why this works
Shows a method instead of a ramble, and tells the interviewer what's coming before you start.
3
State the position, with no hedge
Say it like this
"My position: charge for it. A flat fee, small next to the eventual contract, big enough that nobody signs up out of idle curiosity. The one time I don't charge is when I already know this contractor's own bid mix is exactly what my eval set is missing."
Why this works
PICK rewards a real, stated position, not "it depends," said in the first breath.
4
Name who feels each kind of miss, in real units
Say it like this
"Run every pilot free, and here's who pays: my solutions engineer, about thirty five hours per pilot integrating a contractor's plan library and old bids, for a pilot that six times out of the last nine never produced one completed bid inside the tool. Price it wrong the other way and a contractor like Stonebrace walks, and I never see the forty five percent of their bid book that's modular construction, a project type my golden set barely has fifty examples of."
Why this works
Turns "the stakes are different" from a claim into a number the interviewer can picture.
5
Name the cost asymmetry, plainly
Say it like this
"The free pilot that fizzles is cheap and it's loud. I can see it: a stalled Slack channel, a kickoff call that never got a follow up, thirty five wasted hours sitting right there in the timesheet. Losing the design partner is quiet and it's expensive. I don't get a number for what their modular bids would have taught the model. I find out later, when Bidframe puts a confident price on a modular job for somebody else, and it's wrong, and nothing in the golden set looked like that job."
Why this works
Names which miss is which, instead of leaving "asymmetry" as an unexplained word.
6
Name the kill criteria, with a real number
Say it like this
"Here's what flips it. If a prospect's bid mix sits in a project type where my golden set has under fifty examples, and that type is more than twenty percent of their own annual bid volume, I waive the fee. Stonebrace runs forty five percent modular against a golden set with forty six modular examples. That clears the line easily, so their pilot is free, and I say why out loud on the call, not as a quiet discount."
Why this works
A kill criteria with no number attached is just a feeling. This one is checkable.
7
Close on one line, the trade named
Say it like this
"So the pilot costs money by default, because a fee filters out the pilots that were never going to produce real data anyway. I'm trading a slower sales cycle for pilots that actually hand me failure cases the eval set needs. The one time I skip that trade is when the data I'd lose is worth more than the fee I'd collect."
Why this works
Ends on the line the interviewer remembers, and shows the pick can move if the evidence does.

Let's learn

What does a general contractor actually owe you for six weeks with a tool they haven't decided to trust yet?

Bidframe is a tool a construction-tech company called Trussgate sells to general contractors. It reads a project's plans and spec sheets, checks them against thousands of past bids, and hands an estimator a starting price for every line item, along with a note on which numbers it's sure about and which ones need a second look before anyone submits them.

For its first two years, Trussgate ran every pilot free. No fee, no contract, just a shared folder and a kickoff call. It worked, because in a market with no track record yet, free was the only way to get a contractor to open the folder at all.

It stopped working the way it used to. Of the last nine free pilots, six never had a single real bid run through the tool. A contractor's estimating lead would agree on the call, sound genuinely interested, and then go back to the bids that were actually due that week. The free tool sat there, untouched, until the six weeks ran out.

Here is the turn. The wasted sales calls are not the real problem. The real problem is what happens when a team reacts to that by protecting itself the wrong way: charging everyone the same fixed price with no room to think, and nearly losing a contractor whose bid mix was exactly the kind of project the model had almost never seen.

Pilots that never ran a real bid, free versus paid
100% 50% 0 67% 12% Free (6 of 9) Paid (1 of 8)
Same six-week window, same solutions-engineering setup on each side. The only thing that changed was whether the contractor had money in the pilot.

At its worst, this costs more than thirty five wasted hours here and there. A team that reacts to free-pilot fatigue by charging a flat, uniform fee with no lever to pull can end up shipping Bidframe to more contractors with a tool that sounds confident everywhere, while quietly guessing wrong on any project type it was never actually trained to see.

We did not almost lose thirty five wasted hours. We almost lost the forty five percent of a contractor's bids that would have taught Bidframe what a modular job actually looks like.
The decision I would take back Right after a free pilot came back with zero usage, Trussgate set one flat fee for every new pilot, no exceptions, and no way to check whether a given prospect's work filled a real gap in the golden set. The rule protected against tourists. It could not tell a tourist from the one contractor whose data actually mattered.
Knowledge spark: what is a golden set? A set of real, hand-checked examples used to test whether the model is actually right, not just confident. A golden set with almost no modular-construction bids in it means nobody can measure how good Bidframe really is on a modular job, only guess.

What I would leave alone. Support for a contractor who has already signed should still be fast and free, this is only about the unsigned trial window. A tiny, self-serve trial, upload one project, no calls, no integration, can also stay free forever, because it costs almost nothing to run and it was never going to fill the golden set anyway.

The lesson: a price on a pilot was never there to make money on the pilot. It is there to sort the contractors who will actually run real bids through the tool from the ones who signed up because it was free and it was Tuesday.

Hand sketched two panel comparison. Left panel, small green box, labeled free with no skin in the game, captioned wasted onboarding hours, a pilot channel that goes quiet. Right panel, larger red orange box, labeled priced too high, captioned a design partner walks, and their bid types never reach the golden set.
One side of this costs a timesheet entry. The other side costs a whole project type nobody ever measures the model against.

The week Stonebrace almost said no

the short version sits above. read on for what it actually sounds like when a contractor pushes back on a fee, and what almost got missed.

Every Tuesday morning for two years, Obiora Uzoma ran the same call at Trussgate: a kickoff, a shared folder link, a promise to check back in six weeks. He runs pilots for Bidframe, and for most of that time the process barely changed, because it barely needed to.

In the early years, free pilots were the entire way Trussgate got anyone to try Bidframe at all. The first dozen contractors who tried it free went on to sign real contracts, because in a market with no track record, free was the only door anyone would walk through.

As Bidframe's name spread, the pilots stopped meaning the same thing. Some contractors wanted a free peek and nothing more. Some had an in-house estimator quietly benchmarking their own numbers against Bidframe's, with no plan to ever pay for it. And some simply meant to try it and then got busy the day the pilot started, the way anyone does with something that cost them nothing to say yes to.

The sixth of nine, a general contractor called Colbeck Builders, agreed enthusiastically on the kickoff call and then never opened Bidframe once in six weeks. At week six, Obiora pulled the usage log. Zero projects uploaded. Nothing to point to, good or bad.

That week, Obiora set a flat fee: nine thousand dollars for any new pilot, applied the same way to everyone, no exceptions. It worked exactly as intended. Pilot volume dropped by more than half. Of the pilots that did sign up and pay, seven of eight ran at least one real bid through the tool inside six weeks, and three converted to paying contracts.

Then Stonebrace called.

Stonebrace Construction is a mid-size general contractor with a real specialty: nearly half its work is modular and prefabricated building, an unusual mix for a company its size. Radek Marchetta, Stonebrace's chief estimator, had heard about Bidframe from a competitor and sounded interested from the first minute of the call. He was also plain about the fee. "We don't pay to test-drive software," he said. "Prove it works, then we'll talk about money."

Following the flat rule exactly as written, Obiora held the line. Radek said he'd think about it. Two weeks went by with nothing back.

Nobody at Trussgate had done anything wrong. The rule was simply not built to notice the one time it was the wrong rule.

What broke the silence was a routine check, not a change of heart. Trussgate ran a quarterly audit of its own golden set, the collection of hand-checked bids used to grade Bidframe's accuracy. Modular and prefabricated construction made up about 46 of roughly 1,200 examples in it, under four percent, next to nothing next to the amount of ordinary stick-built commercial work in there. Obiora already had the bid list Radek had shared during the sales call sitting in his notes. Stonebrace's own annual bid volume ran about 45 percent modular.

Obiora went back to Radek. Not with a quiet discount dressed up as a favor, but with the actual reason named out loud: "Your bid mix is exactly the thing our model is thinnest on. I'll waive the fee for this pilot, but I need your estimators to flag every place Bidframe's number looks wrong, especially on the modular jobs." Radek agreed, mostly because it read as a real trade, not a favor he had to feel grateful for.

Hand sketched two panel comparison. Left panel, small green box, labeled free with no skin in the game, captioned wasted onboarding hours, a pilot channel that goes quiet. Right panel, larger red orange box, labeled priced too high, captioned a design partner walks, and their bid types never reach the golden set.
The same picture from Section 1, worth holding onto here. Stonebrace was standing in the box on the right until somebody checked the golden set.

Stonebrace ran 22 real bids through Bidframe over the six weeks. Radek's team corrected nine of them by hand. Seven of those nine corrections were modular jobs, where Bidframe's line item price had come in more than 15 percent off. Every one of those nine corrections became a new golden-set example. Stonebrace signed a contract after the pilot ended. For the next twelve prospects who came in without that same kind of data gap, the nine thousand dollar fee stood exactly where it had, and the no-show rate on paid pilots held close to what it had already been.

The old decision, told as a memory of an actual afternoon: the flat, no-exceptions fee was set in a fifteen-minute call with Trussgate's head of sales, right after the Colbeck usage log came back at zero. "No exceptions" was the exact phrase used, because the team had just watched a looser policy quietly turn into forty free pilots a quarter. It made complete sense in that room. It stopped working the moment it also blocked the one contractor whose bids the model actually needed.

One design protects you from every prospect who wants a favor. It cannot, on its own, tell the difference between a favor and the one contractor you actually need to pay to keep.

What I would tell myself, back in that fifteen-minute call: you built a filter for commitment. You forgot to build a lever for the one thing you were actually pricing the pilot to get.

PICK, run against the Stonebrace fee

this is a commit-then-defend question, so PICK carries the weight, not a list of pros and cons.

P, position. Charge a flat fee for every new pilot by default, small next to the value of a signed contract, applied the same way to every prospect. Waive it only when the prospect's own bid mix sits in a project type the golden set is thin on.
I, impact. A free pilot that fizzles costs Trussgate's own team, about 35 solutions-engineering hours per no-show, visible in a timesheet within weeks. A design partner lost over price costs the product itself an entire project type's worth of real failure cases, invisible until Bidframe recommends a confident, wrong number on that same kind of job for somebody else.
C, cost asymmetry. The wasted free-pilot hours are cheap and loud: a stalled channel, an empty usage log, a number anyone can look up. The lost design partner is quiet and expensive: nobody logs the modular bids that never got seen, and the gap only shows up later as a bad recommendation on a live job, discovered by a customer instead of caught by an eval set.
K, kill criteria. Waive the fee when a prospect's bid mix sits above 20 percent in a project type where the golden set has under 50 examples. Stonebrace ran 45 percent modular against 46 golden-set examples, well past both numbers. Below that line, the fee stands, because most prospects are simply more of the ordinary work the model already handles well.

The alternative I ruled out: keep every pilot free forever, on the idea that more trial volume beats a filtered, paid funnel. That was Trussgate's actual first two years, and nine tries at it already proved the problem: six of nine free pilots produced zero real bids and zero usable data, only a longer list of contacts who'd once said yes on a call.

This is also the exact spot where the answer has to stay AI shaped, not just a pricing story. The real risk sitting behind all of this is Bidframe recommending a confident number on a job type it has barely seen, and an estimator trusting it without a second look. The guardrail already lives in the product, not just the pilot terms: Bidframe never auto-fills a line item below a calibrated match-confidence bar against the golden set, it routes that number to the estimator for manual review instead. During a pilot, how often that flag fires on a new project type is itself a live reading of a coverage gap, which is exactly what happened with Stonebrace's modular jobs.

The trade being accepted Charging by default means a slower sales cycle. Some prospects who'd have said yes to free say no to nine thousand dollars, and the pilot pipeline shrinks because of it. That's a real cost, traded on purpose for pilots that hand back usable failure cases instead of a shared folder nobody opened.
Where the waiver line sits, against real prospects
Below the line, the fee stands
Above the line, the fee gets waived
0% 50% 100% 20% waiver line 6% Anderfeld Builders 9% Colbeck Builders 14% Vosloo Construction 45% Stonebrace Construction
Colbeck Builders sat under the waiver line too, at 9 percent, which is worth noticing: the flat fee was the right call for Colbeck. It was only ever wrong for the one prospect sitting well above the line.

Try it somewhere else, at an insurance MGA

Havelock Analytics builds Riskline, a copilot that reads a commercial insurance submission package and recommends whether to quote the policy, and at what price, for the managing general agents who write that business.

Solvei Kettlewell runs pilots for Riskline the way Obiora runs them for Bidframe. Every new MGA pays a modest monthly fee to pilot Riskline against its own submission flow, by default, no exceptions until the golden set says otherwise.

P. Charge a flat monthly pilot fee for every MGA, by default. Waive it only for an MGA whose book leans heavily into a coverage class Havelock's golden set is thin on.
I. A free pilot that fizzles costs Havelock's underwriting reviewers time on setup calls that go nowhere. A paid MGA lost over price costs an entire risk class's worth of real accept-or-decline examples, the kind that only shows up when someone actually underwrites that class.
C. The wasted setup hours are cheap and visible, caught within weeks. A missing risk class is quiet and expensive: it surfaces later as a bad quote recommendation on a real policy, in a class nobody had enough examples of to catch it sooner.
K. Waive the fee when an MGA's book sits above 25 percent in a coverage class where Havelock's golden set has under 40 examples. Fenmore Underwriters writes about 30 percent environmental liability against a golden set with only 31 examples of that class, clearing both numbers, so its pilot runs free.

What I would leave alone, at Havelock An MGA whose book is almost entirely standard commercial property never needs this exception. The golden set already has thousands of examples there, and a fee that filters for commitment costs that MGA nothing it wasn't already willing to pay for a real product.

Swap the trigger and it still runs.
Speed: an interviewer cuts you off after ninety seconds. Skip straight to the position and the kill number: charge by default, waive only above the coverage-gap threshold.
Cost: say the solutions-engineering team can't staff the extra onboarding calls a paid pilot needs without cutting support elsewhere. The pick doesn't move, the fee is exactly what buys that staffing time back, by filtering out the pilots that would have wasted it anyway.
The model got better, for real: suppose Bidframe's accuracy on modular jobs already matched its accuracy everywhere else. That's the actual evidence that flips the kill criteria off, once modular coverage clears the bar, Stonebrace's bid mix stops being scarce data and the fee goes back to standing like anyone else's.

Where people run it wrong.
Treating a waived fee as a favor for being nice, instead of naming the specific data gap it's buying, so every later prospect assumes they can ask nicely too.
Keeping every pilot free forever because a slower sales cycle looks bad on a pipeline chart, while the real cost is a folder full of pilots nobody ever opened.
Setting one fee at launch and never revisiting it, so a number that once filtered for commitment quietly becomes too cheap to filter anyone, or too high for the exact segment you need.

How to use it live. Say the reframe out loud before answering with a rule of thumb. "Give me a second, I want to check whether this prospect's data is scarce for us or just more of what we already have." That's true, it's already the kill-criteria step, and it buys you the time to find the real number instead of reciting charge always or never charge.

Flashcards (click a card to flip it)

1 · THE FRAMEWORK
Which framework fits this question, and what's the hardest step to nail?
Tap to flip
ANSWER
PICK, for a tradeoff. The hardest step is C, the cost asymmetry: naming why a lost design partner's missing data costs more than a fizzled free pilot's wasted hours.
2 · THE PERSON
Who is this answer about, and what does he already do well?
Tap to flip
ANSWER
Obiora Uzoma, who runs pilots for Bidframe at Trussgate. Two years into the job, and the one who actually checks a usage log instead of trusting a good kickoff call.
3 · THE OLD POLICY
What did Trussgate stop doing once free pilots stopped filtering for anything?
Tap to flip
ANSWER
Running every pilot free with no exceptions asked for or granted. After six of nine free pilots showed zero real usage, Trussgate switched every new pilot to one flat paid fee.
4 · THE ASYMMETRY
What are the two ways to get this pricing decision backwards, and who gets hurt by each?
Tap to flip
ANSWER
Running every pilot free hurts Trussgate's own team: wasted onboarding hours on prospects with no skin in the game. Pricing every pilot the same, with no waiver lever, hurts the product itself: it can lose the one design partner whose data the golden set actually needs.
5 · THE POSITION
State the pick in one sentence, the way you'd say it out loud.
Tap to flip
ANSWER
Charge a flat fee for every pilot by default. Waive it only when the prospect's own bid mix would fill a real hole in the golden set.
6 · THE NUMBER
Fill in the blank: of the last nine free pilots, ______ never ran a single real bid, versus ______ of eight once the flat fee began.
Tap to flip
ANSWER
Six of nine free; one of eight paid. The fee cut the no-show rate from about 67 percent to about 12 percent.
7 · THE KILL CRITERIA
What evidence flips the pick from charging Stonebrace to waiving the fee?
Tap to flip
ANSWER
A project type under 50 golden-set examples, where the prospect's own bid mix sits above 20 percent of their annual volume. Modular sat at 46 examples and Stonebrace ran 45 percent modular, so the fee got waived, and named as a trade, not a favor.
8 · THE TRANSFER
Section 4 runs PICK again on a different product. Which one, and where does the position land there?
Tap to flip
ANSWER
Riskline, an underwriting copilot at Havelock Analytics, sold to insurance MGAs. Same rule: charge a monthly pilot fee by default, waive it only for an MGA whose book fills a thin coverage class in Havelock's golden set.

Check yourself Score: 0 / 0

Multiple choice
1. Which of these is the actual reasoning behind charging for the Bidframe pilot instead of running it free for everyone?
  • A. Charging proves the contractor can afford Bidframe once the pilot ends.
  • B. A fee filters for contractors who will actually run real bids through the tool, which is what produces usable pilot data.
  • C. It stops Bidframe from ever recommending a wrong number during the trial.
  • D. It guarantees the contractor signs a full contract afterward.
Show hint
Check what actually happened to the free pilots that never converted.
Show answer
B. Six of nine free pilots never ran a real bid at all. The fee was never about proving budget or guaranteeing a sale, it was about buying pilots that produce real, usable data instead of a shared folder nobody opened.
Fill in the blank
2. Fill in the blank: of the last nine free pilots Trussgate ran, ______ never had a single real bid run through the tool.
Show hint
It's the number the flat fee decision was made right after.
Show answer
Six. Six of nine, about 67 percent, produced zero usable data, which is what pushed Trussgate to switch every new pilot to a flat paid fee.
True or false
3. True or false, with why: once Trussgate switched to a flat paid fee for everyone, the right move for Stonebrace was to hold the line and charge the same fee as any other prospect.
  • True
  • False
Show hint
Check Stonebrace's bid mix against the golden set's modular coverage.
Show answer
False. Stonebrace's bid mix, 45 percent modular, sat well above the 20 percent waiver line, and the golden set had only 46 modular examples, under the 50-example bar. This is exactly the case the kill criteria exists for, so the fee got waived and named as a trade.
Short answer
4. What old decision does this answer take back, and why did it make sense when it was first made?
Show hint
Look for the fifteen-minute call, not a dial anyone could just turn back up.
Show answer
Model answer: The flat, no-exceptions pilot fee, set right after Colbeck Builders' pilot came back with zero usage. It made sense in that moment because exceptions were exactly how the earlier free-for-everyone policy had quietly turned into forty free pilots a quarter. It stopped working once it also blocked the one prospect whose bid mix the golden set actually needed.
Short answer, apply it yourself
5. Think of a free trial or free tier you've used yourself. What would the paid version have filtered for, if the company had charged for it from the start?
Show hint
Look for the difference between someone who signed up out of curiosity and someone who actually needed the product.
Show answer
Model answer: "A project management tool I tried had a free 14-day trial with no card required. Almost everyone on my team signed up, and most never opened it past day two. If it had cost even ten dollars up front, the trial would have filtered for people who actually meant to switch tools, not people who clicked a button because it cost nothing to click."
Short answer, the number question
6. If Trussgate's golden set already had 400 modular examples instead of 46, would Stonebrace's fee still get waived under the kill criteria? Show the reasoning.
Show hint
Check what the waiver rule is actually measuring, coverage or convenience.
Show answer
Probably not. The waiver rule targets a real gap, a project type under the 50-example bar. At 400 examples, modular construction would already be well covered, so Stonebrace's 45 percent share wouldn't fill a hole anymore, it would just be more of what Trussgate already has. The fee would stand, the same as it does for any other paid pilot.
Before you close the answer
Why this works
Tests whether you can name a real cost asymmetry with units attached, not just say "charge for pilots" and stop. The kill criteria is the actual proof you can move off your own position when the evidence says so.
Follow-up traps
"Isn't waiving the fee for Stonebrace just favoritism dressed up with a number?" Response: no, because the reason was named out loud on the call and tied to a specific, checkable gap, under 50 golden-set examples against a bid mix over 20 percent. A quiet discount would be favoritism. A named trade with a threshold is a policy.

"What stops every prospect from just claiming their work is unusual to get a free pilot?" Response: the threshold is checked against Trussgate's own golden set, not the prospect's claim. Anderfeld and Vosloo also pitched real projects and still landed under the waiver line, so the fee stood for both of them.
If pressed
Bidframe's line-item auto-fill sits behind a 72 percent match-confidence bar against the golden set. Below that bar, the number never auto-fills, it routes to the estimator for manual review, and during a pilot, how often that flag fires on a new project type is tracked separately as a live coverage signal, which is what first pointed Obiora at the modular gap before the quarterly audit confirmed it.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more