InterviewAdvancedQuality, Cost & Token Economics / Cost modeling and unit economics / #25
Walk me through the unit economics of an AI product I name.
BOUND · unit economics, worked live
Larkspire, Quenwick's support chatbot, answers about ninety thousand customer conversations a month. Its own bill is almost nothing. The number that actually decides whether it's worth running lives somewhere else entirely.
The direct answer
Split the cost into two paths and weight them by containment rate: a small AI cost every conversation pays, plus a much bigger human cost that only escalated conversations pay. For Larkspire, ninety thousand conversations a month, sixty one percent contained by the bot alone, an AI cost near a penny a conversation, and two dollars eighty cents for every conversation a person has to finish, land the honest monthly bill near $103,380, in a range of $88,260 to $118,500. The assumption that swings that total the most is containment rate, not the price of a single AI call: raising containment from forty two percent to sixty one percent nearly doubled the AI's own cost line and still cut the total bill by about $47,000 a month.
Do this, in order
Split the cost into two paths, a small AI cost every conversation pays and a bigger human cost only escalations pay, weighted by containment rate.Why: containment rate, not the price of a single AI call, is what the whole monthly bill actually turns on.
Own every number in both paths and say where it came from: volume, containment rate, AI cost per conversation, and loaded agent cost per escalation.Why: a number nobody can trace to a source is a guess with a dollar sign on it.
Give the total as a range built off containment's real uncertainty, not one confident figure.Why: containment moves month to month with the product catalog and the questions customers actually ask, and one number hides how much that shift costs.
Sanity check the total against running the same volume with no AI at all.Why: $103,380 looks fine on its own and only means something once it's checked against $252,000 for an all human team handling the same ninety thousand conversations.
Track containment rate itself with a fixed weekly eval set, not just the finance total each month.Why: a knowledge base going stale or a new policy the bot was never told about drops containment quietly, and the bill doesn't show it until a full billing cycle later.
Don't chase the AI cost line down to save pennies once containment is doing the real work.Why: paying more for a better model raised Larkspire's own token spend by about $450 a month and cut the total bill by $47,000, because the model's quality bought containment, not the other way round.
How to answer this, stage by stage
Nobody is grading whether you know what a chatbot costs to run. They're grading whether you'll build the whole equation live, on a product you didn't pick, and find the number that actually decides it.
1
Scope it to one product and the two people who own the numbers
Say it like this
"Let's ground this in one product, since you named 'an AI product' without picking one for me. I'll use Larkspire, the support chatbot Quenwick, a mid-size outdoor gear retailer, runs on its help center. Dominika Vantwell owns its cost model, and Yseult Sorrengale is the finance partner who signs off on what it costs to run."
Why this works
Naming a real product turns an open prompt into arithmetic instead of a lecture on what unit economics means in general.
2
Say your structure out loud before touching a number
Say it like this
"I'm going to split this into two paths: what every conversation costs the AI to attempt, and what only the escalated ones cost once a person takes over. Then I'll weight those by containment rate, check the total against a range, sanity check it against running the same volume with no AI at all, and say which assumption would move the answer most."
Why this works
States a method before a figure, which is what a real estimator does instead of a guess with a chart pasted behind it.
3
Name the two paths as an equation
Say it like this
"Monthly cost equals AI cost times every conversation, plus human cost times only the conversations that get escalated, plus a small fixed line for hosting the knowledge base and watching the pipeline."
Why this works
Naming both paths before a single figure is what keeps 'AI cost' from being treated as the whole story.
4
Own every number and where it came from
Say it like this
"Larkspire handles about ninety thousand conversations a month. It resolves sixty one percent on its own, so thirty nine percent go to a person. The AI attempt costs about a penny a conversation, every conversation, since it always takes the first pass. A person finishing an escalated conversation costs about two dollars eighty cents, loaded, for around six minutes of a support rep's time."
Why this works
A number nobody can trace to a source is a guess wearing a currency symbol.
5
Solve the equation and give it as a range
Say it like this
"AI cost across all ninety thousand conversations runs about $990. Escalations, thirty nine percent of ninety thousand at two eighty each, run about $98,280. Add roughly $4,110 for hosting and monitoring and the honest number lands near $103,380 a month. Containment moves with the product and the season, so the real range is about $88,260 to $118,500."
Why this works
A single point estimate hides how much the total depends on a rate that shifts month to month, not a fixed price.
6
Sanity check it against running the same volume with no AI at all
Say it like this
"Ninety thousand conversations handled entirely by people, no chatbot at all, costs about $252,000 a month at that same two dollars eighty a conversation. Even at the worst end of Larkspire's range, $118,500, that's still less than half. What's actually interesting is how little containment it would take to break even at all, about two percent."
Why this works
A total that only compares to itself never tells you whether it's actually a good number.
7
Name the assumption that would move the answer most, and the trade nobody expected
Say it like this
"Containment rate. A few months back, Quenwick's team upgraded Larkspire's model, which nearly doubled the AI cost line, from about $540 to $990 a month. On a budget deck that looks like the wrong direction. It pushed containment from forty two percent to sixty one percent, and that cut the total bill by about $47,000 a month."
Why this works
This is the moment a strong candidate shows they know which number is decoration and which one is the actual lever.
8
Close on the decision, not the arithmetic
Say it like this
"So: two paths, real numbers behind both, a total near $103,380 a month, checked against what an all human team would cost, with containment rate, not the AI's own token price, as the number I'd actually watch every week."
Why this works
Ending on what you'd track, not the last figure you multiplied, is what makes it sound like a system you'd run, not a spreadsheet read aloud.
Let's learn
Every month, about ninety thousand people wrote in to Quenwick's support team asking where an order was, whether a jacket ran small, or how to send back a tent that didn't fit in the trunk.
Larkspire is the chatbot that answers first, before any of those questions reach a person. It reads the order, checks the return policy, and tries to close the ticket on its own. If it can't, it hands off to a support rep, with everything it already found attached.
Before the upgrade Quenwick's team made a few months back, all ninety thousand conversations a month would have cost about $252,000 to staff entirely with people, at roughly two dollars eighty cents a conversation once wages, benefits, and management are counted in.
Knowledge spark: what's containment rate?
The share of conversations the AI finishes on its own, with no person needed at all. Sixty one percent containment means sixty one out of every hundred conversations never reach a support rep.
With Larkspire running at sixty one percent containment, the honest monthly bill lands near $103,380, about a dollar fifteen a conversation once every path is added in. That's real money saved, and it's not the interesting part.
The AI cost line nearly doubled. The total bill dropped by forty seven thousand dollars a month.
The AI's own cost was never the problem, at a penny a conversation it barely shows up next to anything else. The real question was always how many of those ninety thousand conversations still needed a person, and that number moved because Quenwick paid more for a better model, not less.
The build-up: what Larkspire's honest monthly bill is made of
AI cost, every conversationEscalation, human finishes itHosting and monitoring, fixed
The AI line is barely tall enough to see. Escalation, the conversations a person still has to finish, is nearly the whole bill.
The decision that mattered
Quenwick's team swapped Larkspire onto a stronger, pricier model a few months back. That nearly doubled the visible "AI cost" line on the budget deck, from about $540 to $990 a month, which looked like spending going the wrong way until the containment numbers came back.
At its worst, this isn't really about a penny-sized AI cost line at all. If containment ever slips back toward forty two percent without anyone noticing, a stale knowledge base, a new return policy the bot was never told about, the bill creeps back toward $150,810 a month, while everyone keeps nodding at a flat, tiny "AI cost" chart that never told them anything was wrong.
The honest range for Larkspire's blended cost barely moves, $0.98 to $1.32 a conversation. Running the same volume with no chatbot at all sits in a different world, $2.80 a conversation.
What I'd leave alone: the fixed hosting and monitoring line, $4,110 a month, doesn't need this kind of scrutiny. It scales with the size of the knowledge base, not with how many conversations come in or how many get contained, so it stays small and predictable no matter which way containment moves.
The lesson: a cost model can look wrong on one line while being right on the total. The AI cost line got bigger. The whole bill got smaller. Watching only the line that's easy to see teaches you the wrong thing about a product built to hand work off to a person.
Now here is the same thing as a story
Read the story below when you want to feel why a bigger AI bill was actually the fix, not just be told that it was.
Dominika Vantwell built Larkspire's routing pipeline the year Quenwick decided its support inbox had gotten too big for the team it had. She'd shipped internal tools before, nothing customer-facing, and Larkspire was the first thing she built that a shopper would actually talk to.
At launch, on the cheapest model tier available, Larkspire cost about $540 a month to run and resolved forty two percent of Quenwick's roughly ninety thousand monthly conversations without a person. That felt like a fair trade for the price: nearly half of a huge queue, gone, for less than the cost of one part-time hire.
For the better part of a year, that number held. Dominika's monthly dashboard showed one line, "AI cost," and it barely moved. Leadership liked a chart that stayed flat, and a flat chart is an easy thing to stop looking at closely.
The habit that faded wasn't dramatic. First, "AI cost per conversation" became the one number the weekly ops review actually looked at, since it was small and simple and always told the same good story. Then nobody built a second line for what escalated conversations cost, since those ran through the same support queue Quenwick had always used, tracked in a system finance already owned and never thought to connect. Then, without anyone deciding it on purpose, keeping that one line flat quietly became a real design goal on its own.
Once a quarter, Quenwick's finance team runs a routine spend audit, flagging any vendor line that crosses $500 a month for the first time. Larkspire's AI line had sat under that mark for a year. Then Dominika's team ran a small pilot swapping in a stronger model, and the line ticked past it. It landed on Yseult Sorrengale's desk with everything else that quarter that had grown.
Yseult didn't assume the bigger number was a problem. She asked Dominika for the number that actually mattered: not what Larkspire cost, but what Quenwick's whole support operation cost with Larkspire running, against what it would cost without it. Dominika spent an afternoon pulling both numbers for real, instead of the one line the dashboard had always shown.
The AI cost line nearly doubled. The total bill dropped by forty seven thousand dollars a month.
The real risk was never that Larkspire's AI line might grow. It was that a flat line had been quietly protecting a containment rate stuck at forty two percent for a year, while every conversation below what a better model could reach was still going straight to a person, at two dollars eighty cents, ninety thousand times a month.
Quenwick never really had an "AI cost" problem. It had an escalation problem wearing an AI cost line as a disguise, and the two only looked separable because nobody had ever put them on the same dashboard.
The decision Dominika would take back happened in a much smaller room, the week Larkspire launched, when three people picked which model tier to start on. Choosing the cheapest one wasn't wrong. At $540 a month against a total cost nobody had modeled yet, it was the careful choice. What nobody did in that room was write down that "AI cost" and "total cost" were two different numbers, and only keep watching one of them.
Run that quarterly audit again, with a blended dashboard in place from month one instead. The AI line still crosses $500 eventually, exactly like before. Except right next to it sits a second number, cost per conversation across both paths, and it's already been falling for months by the time the audit runs, from a dollar sixty eight down to a dollar fifteen. The audit finds a line that grew next to a total that shrank, and nobody spends an afternoon working out whether that's good news, because the dashboard already says so.
One dashboard showed the AI spending money. The other showed the whole operation saving it, and they were both measuring the exact same chatbot the entire time.
What Dominika would tell herself, back in that first small room: the cheapest model wasn't the wrong call at launch. It was a number that never got a partner number sitting next to it, so for a year, the only thing anyone could see was the half of the story that looked like a cost.
BOUND, worked start to finish: the two paths behind Larkspire's number
Not a story question wearing a framework's clothes. This is a live estimation build on a product the interviewer names, and BOUND is what keeps "AI cost" from swallowing the whole answer.
BBreak it down. What are the two paths?
Monthly cost equals AI cost, paid by every conversation since the bot always takes the first pass, plus human cost, paid only by the conversations that get escalated, plus a small fixed line for hosting the knowledge base and watching the pipeline. Two paths, weighted by containment rate, not one blended guess.
Say both paths before naming a figure, or "AI cost" quietly becomes the whole answer.
OOwn the numbers. Where did each one come from?
Ninety thousand conversations a month. Containment rate: sixty one percent, up from forty two percent before a recent model upgrade. AI cost: about a penny a conversation, every conversation, on the current model. Human cost: about $2.80 a conversation, loaded, for roughly six minutes of a support rep's time on an escalated conversation. Fixed line: about $4,110 a month for hosting and monitoring.
This is also where the rejected alternative sits, see below: staying on the cheaper model to keep the AI cost line flat.
UUse a range, not one number.
Containment realistically moves between fifty five and sixty seven percent month to month. That puts the honest total between about $88,260 and $118,500, with a best estimate near $103,380, or about $0.98 to $1.32 a conversation.
A single confident number is exactly what let a flat AI cost line sit unquestioned for a year.
The honest answer was never one number. It runs from about $0.98 to $1.32 a conversation, well under half of what the same volume costs with no chatbot at all.
NNail the sanity check. Does the number survive being compared to something real?
Ninety thousand conversations handled entirely by people, no chatbot at all, costs about $252,000 a month at $2.80 each. Even the worst end of Larkspire's range, $118,500, is still less than half that. The check worth running further: at what containment rate does Larkspire stop paying for itself at all? About two percent. Quenwick is nowhere near that floor, which means the real question was never "should this exist," it's "how much containment is Quenwick leaving on the table."
The hardest step, and the one most answers skip. A total that only compares to itself never tells you whether it's actually a good number.
DDirection. Which assumption would move the answer most?
Containment rate, by a wide margin. Moving it across its realistic range swings the total by about $30,240. A loaded agent cost that moves the same way swings it by about $14,040. Doubling the AI's own token price, the number most people fixate on, swings it by less than $1,000.
Naming the assumption you trust least, out loud, is what a good estimator does that a bad one skips.
What moves Larkspire's bill most, if the assumption behind it is wrong
Biggest swingMedium swingSmallest swing
Containment rate swings the total more than twice as much as agent wages do, and about thirty times more than the AI's own token price. The number worth watching every week was never the model's price tag.
Three things worth stating directly, since this is where the real judgment sits. The alternative Dominika's team lived with for a year, and the one Yseult's audit finally forced a real look at, was staying on the cheaper model to keep the visible "AI cost" line flat. It lost once the full math showed that line was hiding a much bigger one: forty two percent containment meant $146,160 a month in escalations alone, next to $540 in AI spend nobody was worried about. The AI-specific failure worth naming by name is silent containment drift: a knowledge base that goes stale, a new return window or shipping partner the bot was never told about, and containment slips without a single error message or angry customer pointing at it directly. The guardrail is a fixed weekly eval set, about 400 real past conversations replayed against whatever model and prompt version is live, flagging any conversation that used to resolve and no longer does, so a slipping containment rate gets caught before it shows up as a bigger bill a month later. And the trade worth saying out loud is quality against cost: Quenwick accepted a higher, guaranteed AI spend in exchange for a materially higher containment rate, on the condition that "contained" still means the answer actually clears Quenwick's own accuracy bar on that weekly eval set, not just that the conversation closed without a person. A model that closes more tickets by guessing more confidently isn't containment, it's a returns dispute waiting to happen.
And if you want to be sure it really works, try it somewhere else
Same five letters, a study guide generator instead of a support chat, and this time the lever isn't how many conversations need a person, it's how many guides a student throws away and asks for again.
Fernhollow Learning is an online tutoring platform. Loomcard is the tool inside it that turns a student's course reading into a one-page study guide plus a short practice quiz. Corvessa Alcazar runs its cost model.
The build-up: Loomcard generates about 40,000 study guides a month. Each one costs about $0.045 to generate, a longer single pass than a chat reply, since it reads a full chapter and writes a page back. About thirty five percent of guides get regenerated at least once, a student hits "try again" because the guide missed the right chapter or pitched the questions at the wrong level, and each regeneration reruns that same $0.045 cost.
The decision Corvessa would take back
Loomcard's regenerate button shipped free and unlimited from day one, on the assumption regenerations would be rare. Nobody ever built a way to ask a student why they hit it, only that they had.
Total monthly generation cost, base guides plus regenerations, runs about $2,430. Split across Fernhollow's twelve thousand monthly active students, that's about twenty cents a student, next to nothing against a fifteen dollar subscription.
Same method, different lever: for Larkspire, the swing was containment rate against a two dollar eighty escalation cost, a real dollar swing worth tens of thousands a month. For Loomcard, the dollar swing barely exists, twenty cents a student either way it goes. The number actually worth watching there is what a thirty five percent regeneration rate is standing in for: a guide bad enough on the first try that a paying student had to ask twice, which is a churn question wearing a token-cost question's clothes.
Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: split the paths, own the numbers, weight by the rate that decides which path a unit takes, sanity check against the version with no AI at all.
Cost: there's no budget this quarter for both a Larkspire model upgrade and a Loomcard regeneration-reason survey. Larkspire wins first, its dollar swing is $47,000 a month against Loomcard's entire $2,430 bill; the regeneration signal waits.
The model got better, for real: say the underlying model gets a real quality bump for the same price. For Loomcard that quietly lowers the regeneration rate, which barely moves the dollar total but might move the churn number a lot. For Larkspire, the same free upgrade could lift containment and lower cost at once, a rarer double win worth catching.
Where people run it wrong.
They track the AI's own per-call cost as the number that matters, and never build the second path, human cost or churn risk, sitting right next to it.
They assume a small dollar swing means small stakes, when the real cost, a lost customer, a wrong answer nobody caught, is hiding behind a rate, not a price.
They set the "good enough" model tier once at launch and never revisit it once the blended math actually gets checked.
How to use it live. Ask the real question before quoting a number: "before I give you a monthly total, can I confirm this product has two real cost paths, or just one?" That's a genuine diagnostic question, and it buys you thinking time instead of reciting a number nobody in the room could rebuild from scratch.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework is this, and what's its one job?
Tap to flip
ANSWER
BOUND: show the arithmetic, own the assumptions. Built for a live unit-economics build on a product named on the spot, not a story about a habit.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Dominika Vantwell, who owns Larkspire's cost model at Quenwick, working with Yseult Sorrengale, the finance partner who flagged its AI line during a routine spend audit.
3 · THE BLIND SPOT
What number did Quenwick's dashboard never show next to the AI cost line?
Tap to flip
ANSWER
The blended cost per conversation across both paths, contained and escalated. Only the AI's own per-conversation cost was ever tracked, so a shrinking total looked identical to a growing one on the one chart anyone watched.
4 · THE EQUATION
What are the two paths that make up Larkspire's real monthly cost?
Tap to flip
ANSWER
AI cost, paid by every conversation, plus human cost, paid only by escalated conversations, weighted by containment rate, plus a small fixed hosting and monitoring line.
5 · THE OLD DECISION
What decision would Dominika take back?
Tap to flip
ANSWER
Launching Larkspire with one dashboard line, "AI cost," and no second line tracking what the whole support operation actually cost with the chatbot running.
6 · THE NUMBER
Fill in the blank: the honest monthly range ran from $88,260 to $___, with a best estimate near $___.
Tap to flip
ANSWER
$118,500, and $103,380. Upgrading the model nearly doubled the AI line, from $540 to $990, and still cut the total bill by about $47,000.
7 · THE REPLAY
Same quarterly audit, new dashboard, what changes?
Tap to flip
ANSWER
A blended cost-per-conversation number sits next to the AI line from month one. The audit finds a line that grew beside a total that shrank, instead of an afternoon spent working out whether growth was good news.
8 · CROSS-PRODUCT TRANSFER
Section 4 answers this same question again for a different product. Which product, and what's the different lever there?
Tap to flip
ANSWER
Loomcard, a study guide generator at Fernhollow Learning. There the lever is regeneration rate, not containment rate, and the real stakes are churn risk, not a big dollar swing.
Check yourself Score: 0 / 0
Fill in the blank
1. Larkspire's honest monthly range ran from $88,260 to $___, with a best estimate near $___.
Show hint
Look at the U step in the framework recap, right after the equation and the owned numbers.
Show answer
$118,500, and $103,380. That range comes from containment rate moving between 55 and 67 percent; the number that matters most is how far the total moves when containment does.
Multiple choice
2. Why did nearly doubling Larkspire's AI cost line, from $540 to $990 a month, cut Quenwick's total support bill by about $47,000?
A. The cheaper model had a hidden monthly fee that the pricier one removed.
B. The pricier model lifted containment from 42% to 61%, so far fewer of the ninety thousand monthly conversations needed a $2.80 escalation to a person.
C. Quenwick laid off support agents the same month the model changed.
D. The quarterly audit forced a price renegotiation with the model vendor.
Show hint
Look at the O step in the framework recap and the paragraph describing the containment shift.
Show answer
B. Containment rose from 42% to 61%. Every extra percentage point of containment means one more conversation a month that never costs Quenwick $2.80 for a person to finish.
True or false
3. True or false: because Larkspire's own AI cost is under a penny a conversation, the AI cost line is the biggest driver of Quenwick's total monthly bill.
True
False
Show hint
Check the build-up chart in Section 1 and compare the height of the AI cost bar to the escalation bar.
Show answer
False. The AI cost line runs about $990 a month, under 1% of the total. Escalation, at $98,280, is nearly the whole bill.
Short answer, name the rejected alternative
4. What alternative did Dominika's team stick with for a year, and why did it finally get revisited?
Show hint
Look at the O step in the framework recap, where the rejected alternative is named directly.
Show answer
Model answer: Staying on the cheapest model tier to keep the visible "AI cost" line flat and small. It got revisited once a routine quarterly spend audit flagged the AI line crossing $500 for the first time, and Yseult asked for the blended total instead of the one line on the dashboard.
Short answer, apply it yourself
5. Pick an AI feature you use that sometimes hands off to a person or a fallback path. Name the two costs in play and which one you'd expect to actually decide whether the feature is worth running.
Show hint
Think about what the AI attempt costs versus what the fallback path costs, and which of those two actually changes month to month.
Show answer
Model answer: A food delivery app's AI order-issue bot hands unresolved complaints to a human agent. The two costs are the AI's own reply cost, tiny, and the agent's time resolving the handoff, real money. The handoff rate, not the AI's price, is what decides whether the bot is actually saving the company anything.
Multiple choice
6. If Larkspire's containment rate slipped from 61% back to 42% without anyone noticing, what would you expect to happen to the total monthly bill?
A. It falls, since fewer AI calls run when containment drops.
B. It rises to roughly $150,810 a month, since escalations, at $2.80 each, climb back toward the pre-upgrade level.
C. It rises only slightly, since the fixed hosting line is the main driver of the total.
D. It stays the same, since containment rate only affects customer satisfaction, not cost.
Show hint
This is exactly the biggest bar in the D step's sensitivity chart. Recompute the escalation cost at 42% containment.
Show answer
B. At 42% containment, 58% of ninety thousand conversations need a person at $2.80 each, which alone runs $146,160, plus the AI and fixed lines, landing near $150,810 a month, the old pre-upgrade total.
Before you close the answer
Why this works
Tests whether you can build a full unit-economics model live, on a product you didn't choose, and catch that the cost that decides everything usually isn't the AI's own line. Most candidates stop at the token price and call it done.
Follow-up traps
"If the AI's own cost is basically free, why not just run every conversation through it twice to double check?" Response: running it twice doesn't change containment, since the model's own limits, not a second look, are what caps how many conversations it can resolve. What actually raised containment was a better model, not a repeated one.
"Couldn't Quenwick just loosen the escalation threshold so the bot resolves even more, and get the savings without a pricier model?" Response: loosening the threshold blind pushes weaker answers into the "resolved" bucket without raising real quality, and a wrong "resolved" answer costs more downstream, a return dispute, a refund fight, than an honest escalation ever would. Containment has to be earned by a better answer, not declared by a looser bar.
If pressed
The containment eval gate isn't a single pass rate checked once. It replays a fixed weekly set of about 400 real past conversations against whatever model and prompt version is currently live, and flags any conversation that used to resolve and no longer does, catching a stale knowledge-base entry or an unindexed policy change before it shows up as a bigger bill a full billing cycle later.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
