Explain how to negotiate around usage-based pricing uncertainty.
LoadPilot AI sells a load-matching tool to freight brokers, billed per API call. Callum Bright runs vendor and procurement decisions at Haulwright, a regional freight brokerage moving about 1,200 loads a month.
- Push for pricing per completed match, not per raw call.Why: a completed match is the outcome you're paying for; a call count is a proxy that swings with the model's own uncertainty.
- Negotiate a hard retry ceiling per match, billed to the vendor past that point.Why: retries mean the model needed more tries to find a good answer, not that your load volume changed.
- Model your own worst case before signing, not just the sales demo's average case.Why: the average case is what gets sold; the worst case is what gets billed.
- Ask for a monthly cost cap or a true-up credit past an agreed ceiling.Why: it turns an open-ended bill into a bounded one you can actually plan a budget around.
- Track calls-per-match as its own number every month, not just the total invoice.Why: it's the number that moves before the bill does, so it tells you trouble is coming.
How to answer this, stage by stage
Nobody is scoring you on whether you can say "negotiate a discount." They're scoring whether you can show the arithmetic behind why the bill moved, and fix the actual variable.
Let's learn
The laptop Callum Bright uses for vendor invoices has a cracked hinge and a spreadsheet tab that never closes. Say a freight brokerage buys an AI tool that matches loads to trucks and drivers automatically, billed by the API call it takes to find each match.
Before it, Haulwright's dispatchers matched about 1,200 loads a month by phone, roughly 20 minutes of calling per load. LoadPilot AI matches most loads in under a minute, billed at 18 cents a call, with about 6 calls per completed match under normal conditions: search, rank, confirm, recalculate the route, check for exceptions. The first few months, the bill ran close to 1,300 dollars a month, cheap against the dispatcher hours it replaced.
Here's the turn: the extra calls themselves were never the real problem. An extra call costs 18 cents. The real problem was what happened once peak season hit and drivers started cancelling more often: the model needed more attempts, more re-ranks, more re-matches, to land a good pairing. And nobody at Haulwright noticed the bill creeping, because no single month looked alarming on its own.
At its worst: by month six, the bill had tripled to over 4,100 dollars, on the same load volume, with nobody able to point to the day it happened, because there wasn't one.
What I would leave alone: on a normal week, at six calls a match, this pricing is honestly fine, and cheaper than the phone calls it replaced. Renegotiating the whole contract over one clean, uneventful month would waste time nobody has to spare.
The lesson: usage-based pricing on an AI tool isn't really priced on your volume. It's priced on how many tries the model needs to get it right, and that number is the one thing a vendor's demo will never show you going wrong.
Now here is the same thing as a story
The short version above is what you'd say defending this renegotiation to Haulwright's ownership. Read this one for how quietly it actually crept.
The dispatch floor at Haulwright gets loud around 4pm, when the next day's loads start posting. Callum Bright has run vendor decisions there for five years, and he reads an invoice the way some people read a weather report, looking for the line that doesn't match the season.
LoadPilot AI went live in January. The rep's demo, and the first three invoices, all landed the same way: about 1,200 loads, six calls each, 18 cents a call, a bill just north of 1,300 dollars. Callum stopped scrutinizing the invoice line by line around invoice four. It had been the same number three months running.
Peak season started in April. Drivers cancelled more, routes got reassigned mid-morning, and the model started working harder to land each match, sometimes twenty tries instead of six. Nobody announced this. It just showed up, a few cents at a time, spread across thousands of calls Callum never individually saw.
By May the invoice was 1,890 dollars. By June it was 2,650. Neither jump, on its own, looked like an emergency. A vendor's usage bill goes up and down some months, everyone knows that. It was only in Haulwright's quarterly budget review, comparing six invoices side by side instead of one at a time, that anyone noticed the shape of the line.
Callum went back through six months of invoices and split every bill into base calls against retry overage. The base number, the part tied to actual load volume, barely moved. The retry number was almost the entire increase.
Here's what I'd take back. Haulwright signed a per-call rate based on the sales demo's six-calls-a-match number, without ever asking what a worse month would cost. That was a reasonable read of an honest demo. It stopped being reasonable the moment the season changed and nobody had a ceiling in place to catch it.
I would go back and negotiate a price per completed match, with the retries above a set ceiling billed to LoadPilot instead of Haulwright, before signing anything, not after six invoices quietly added up to three times the original number.
And the part I'd tell myself: we didn't get overcharged. We priced the wrong thing from the start, a call count instead of an outcome, and a call count was always going to move with the model's own confidence, not with our business.
BOUND, in one screenNot "guess a number and defend it." BOUND is what tells you which part of the equation you actually control, and which part you don't.
The recap, one line per letter: break it down is the equation itself, own the numbers is naming where 6 calls and 18 cents actually came from, use a range is best case against worst case instead of one guess, and nail the sanity check is comparing the worst case against the old dispatcher cost it replaced.
Direction. What moves the estimate most. Of every assumption in the equation, the retry rate swings the bill hardest, because it is the one number tied to the model's own uncertainty rather than Haulwright's business.
And if you want to be sure it really works, try it somewhere elseSame four letters, an AI triage vendor billing a telehealth company per minute instead of per call. A different kind of retry.
Briarcliff Telehealth pays its AI triage vendor per minute of AI-handled call audio. Yusuf Ademola manages the vendor relationship there. Mapped onto BOUND: break it down is monthly cost equals calls handled, times AI-minutes per call, times price per minute. Own the numbers is 5,000 calls a month, 4 AI-handled minutes a call under normal conditions, 9 cents a minute, from the vendor's own reporting. Use a range is the honest split: simple, single-symptom calls average 4 minutes, but multi-symptom calls that need repeated clarifying questions can run to 14 minutes, and those make up roughly a fifth of monthly volume in a bad month. Nail the sanity check is comparing the worst-case bill, about 2,700 dollars, against the cost of a human triage nurse handling that same call volume, which would run several times higher.
The old decision here isn't an unmodeled peak season, it's a different reversal: Briarcliff signed a flat per-minute rate because the vendor's demo used only simple, single-symptom calls, and nobody modeled what happens when a caller describes three symptoms at once and the model loops through clarifying questions before it can triage confidently. That made sense when the demo was the only evidence anyone had. It stopped making sense once real multi-symptom calls turned out to be a fifth of monthly volume, not a rare exception.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "price the outcome, cap the retries," and stop.
Cost: there's no leverage to renegotiate a signed contract mid-term. Say so honestly, and start by tracking calls-per-match monthly so the next renewal negotiation has real data behind it.
The model gets better, for real: if a vendor update genuinely cuts the average retry rate, that's the direction analysis doing its job, telling you the ceiling can loosen, not that the tracking can stop.
Where people run it wrong.
They accept the demo's average case as the number that will actually get billed.
They watch the total invoice instead of splitting it into base cost and retry overage.
They negotiate the price per call instead of the thing that actually varies, how many calls one outcome takes.
How to use it live. The moment someone says "usage-based pricing feels risky," ask back: which part of the usage is tied to volume, and which part is tied to the model's own uncertainty? Negotiate the second part separately, because it's the one that moves without your business changing at all.
Flashcards (tap any card to flip it)
Check yourself Score: 0 / 0
Show hint
Show answer
Show hint
Show answer
Show hint
Show answer
Show hint
Show answer
Show hint
Show answer
Show hint
Show answer
"Isn't a retry ceiling just going to make the vendor cut corners on hard matches?" Response: no, because the ceiling is on calls billed to you, not calls the vendor is allowed to make; past the ceiling, the vendor keeps trying at its own cost.
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
- A live AI agent you actually shipped
- A launch decision you can defend under pressure
- An interview-ready portfolio, not more flashcards
More on Evaluating AI vendors as a buyer
- #1 List the ten questions you would ask every AI vendor before a pilot.
- #2 How do you evaluate a vendor's quality claims without running your own eval?
- #3 Design the pilot you would run to evaluate two competing AI vendors.
- #4 What contractual terms matter specifically for AI vendors and not for other software?
- #5 How do you assess a vendor's model dependency and what happens if their provider changes terms?
- #6 Describe the data handling questions you would put to a vendor on behalf of your security team.