CalculationAdvancedAI Opportunity & Model Strategy / Build vs buy vs fine-tune decisions / #6
Describe the total cost of ownership of a self-hosted open model versus an API.
BOUNDowning the hardware only pays off once volume clears a real, calculable line, not on principle
FarSight Radiology runs DraftScan, a feature that drafts a first-pass written report from a radiology image, which a radiologist then reviews and signs off. Bridget Osei is the AI PM modeling whether FarSight should keep paying a vendor per report or bring the model in-house, and Tomasz Halvorsen is a radiologist whose reports are the actual data the comparison runs on.
The direct answer
Self-hosting's total cost is mostly fixed, GPU capacity, an ops engineer, retraining, security review, plus a real upfront build, while an API's cost scales directly with volume. At FarSight's current 60,000 reports a month, the API is cheaper, about 21,000 dollars against roughly 37,700. The two lines cross around 132,600 reports a month. Below that line, buy the API. Above it, and only once volume is actually trending there, self-hosting wins.
Do this, in order
Model self-hosted cost as mostly fixed, and API cost as purely variable, then find where the lines cross.Why: this is the real shape of the comparison, not a single snapshot at today's volume.
Amortize the upfront build cost into the monthly number before comparing anything.Why: skipping it understates self-hosting's true cost and moves the crossover volume in the wrong direction.
Count the ops engineer and the retraining cadence, not just the GPU bill.Why: GPU cost is usually the smallest piece of self-hosting's real total, and it's the one everyone estimates first.
Compare the crossover volume to your actual growth trajectory, not just today's number.Why: the right call today can flip in a year if volume is genuinely climbing toward the crossover line.
Weigh data sensitivity and control alongside the pure cost line.Why: a close cost comparison can still favor self-hosting if keeping data in-house is worth a real premium.
Sanity-check the crossover against a number you already trust.Why: converting it into "about double our current volume" catches an error before a budget meeting does.
How to answer this, stage by stage
Six moves. This is an arithmetic question, so the structure carries more weight than any one number, but the structure still has to be said out loud.
Stage 1
Ask for the one number you can't invent
Say it like this
"Before I model this, one question: what's the real monthly report volume today, and is it expected to grow? I'll model FarSight at 60,000 reports a month, and flag that the whole answer depends on where that number is actually heading."
Why this works
Names the number the entire comparison pivots on, in ten seconds, then keeps moving instead of waiting to be handed it.
Stage 2
State the equation before any numbers
Say it like this
"So the shape is: self-hosted monthly cost equals amortized upfront, plus GPU cost, plus ops engineer time, plus retraining, plus compliance review, most of that roughly fixed. API monthly cost equals volume times price per report, purely variable. Find where the two lines cross."
Why this works
The interviewer can follow every number that comes next, and can challenge one term without derailing the whole model.
Stage 3
Put a stake in each assumption
Say it like this
"API price, 35 cents a report, real multimodal cost for an image plus a drafted report. Self-hosted: 150,000 upfront to fine-tune and stand up serving, amortized over three years, that's about 4,167 a month. Four leased GPUs running continuously, about 7,200 a month. A three-quarter-time ops engineer, loaded, about 10,000. Quarterly retraining at 40,000 a cycle, amortized, about 13,333 a month. Compliance review, 3,000."
Why this works
Every number traces to something checkable, a lease rate, a loaded salary fraction, a retraining invoice, not a guess dressed up as precision.
Stage 4
Run the arithmetic and find the crossover
Say it like this
"Self-hosted totals about 37,700 a month at current volume. The API costs about 21,000 at the same 60,000 reports. Self-hosted's fixed pieces add up to about 30,500 a month, and only the GPU line really scales with volume, at roughly 12 cents a report. Setting the two equal, self-hosted and the API cross at about 132,600 reports a month."
Why this works
Naming the exact crossover volume, not just "it depends on scale," is what separates a real model from a vague nod at economies of scale.
Stage 5
Sanity check against something you already know
Say it like this
"Quick check: 132,600 reports a month is a little over double FarSight's current volume. That's a believable multiple for a scale-dependent crossover, not an order-of-magnitude surprise, and it matches the intuition that self-hosting needs real scale to earn back its fixed cost."
Why this works
Converting the crossover into a multiple of a number you already trust is what catches a modeling mistake before a budget meeting does.
Stage 6
Name the assumption that would break it
Say it like this
"The number I'd want to pressure-test isn't the GPU lease rate, it's the retraining cadence, since it's the single largest fixed cost in the self-hosted total. Cut retraining to twice a year instead of quarterly, and the crossover volume drops to around 96,000, well below FarSight's likely growth path over the next two years."
Why this works
Naming which assumption swings the crossover the most is what separates a real estimate from a spreadsheet nobody pressure-tested.
If you remember one thing
Self-hosting isn't cheaper or more expensive on principle. It's cheaper past a specific, calculable volume, and the interviewer is checking whether you can find that number, not whether you have an opinion about owning your own infrastructure.
Let's learn
Here is the whole model in one block, so the shape is visible before arguing about any single line in it.
Four real cost lines, and the GPU lease is usually the smallest of the four, not the largest.
# inputs, current volume
reports_per_month = 60,000
api_price_per_report = $0.35# image plus drafted report
# self-hosted, monthly
upfront_amortized = $4,167# $150k over 3 years
gpu_lease = $7,200# 4 GPUs, $2.50/hr, continuous
ops_engineer = $10,000# 0.75 FTE, loaded
retraining = $13,333# $40k every quarter, amortized
compliance_review = $3,000 self-hosted total = $37,700 / month
API total = $21,000 / month crossover volume = ~132,600 reports / month
Monthly TCO build-up at current volume, self-hosted versus API
At today's volume, the API is cheaper by nearly 17,000 dollars a month. Retraining and the ops engineer, not the GPU, are the two biggest self-hosted lines.
Before anyone modeled the crossover, the working assumption was that owning the model would obviously be cheaper long-term, since a per-report API fee sounds like the kind of cost that should add up fast. At FarSight's actual volume, it doesn't, not yet.
Self-hosting isn't the cheap option or the expensive one. It's the option that's still waiting for enough volume to earn back its fixed cost.
Both are real options today. Which one is cheaper depends entirely on where FarSight's volume actually sits, not on which one sounds more serious.
Knowledge spark: why does self-hosted cost stay mostly flat as volume grows?
Once the model is trained and the serving stack exists, one ops engineer and one retraining cycle can support a wide range of throughput before either needs to grow. Only the GPU line scales meaningfully with volume, since more simultaneous reports need more machines answering them. That's why self-hosted cost rises slowly while API cost rises in a straight line with every single report.
Monthly cost by report volume, self-hosted versus API, crossover marked
API, scales directly with volumeSelf-hosted, mostly fixed
FarSight sits at 60,000 reports a month today, well left of the crossover. The API stays cheaper until volume climbs past about double where it is now.
At its worst, this cost shows up as a self-hosting decision made on principle, GPU and compliance costs paid every month for a year, while volume never actually reaches the crossover, so the fixed cost never gets earned back at all.
The upfront build is five real steps, not one line item, which is exactly why it's the piece most easily underestimated.
The trap in comparing today's bill instead of the whole line
The actual mistake isn't picking the wrong volume to model, it's stopping at the API's per-report price and never asking what self-hosted TCO looks like at all. A per-report fee feels expensive by itself, 35 cents times 60,000 reports reads as a large number, and that feeling alone pushes some teams toward self-hosting before anyone's modeled whether the fixed costs would actually be smaller.
Cost isn't the only axis. Steady, sensitive volume favors self-hosting even before the crossover volume is reached, since an API's elasticity is wasted on a workload that never spikes.
Bridget's honest number, once the full build-up was modeled, wasn't "self-hosting is expensive" or "the API is cheap." It was a specific volume, about double FarSight's current load, past which the answer flips. That's a genuinely different, more useful answer than either extreme.
BOUND, in one screen
This is an estimation question, so the framework is BOUND. There's no habit to trace here, the entire answer is a cost model, and forcing a FLIPS-style story onto it would bolt a parable onto a spreadsheet that doesn't need one.
B
Break it down.
Self-hosted monthly cost equals amortized upfront, plus GPU lease, plus ops engineer time, plus retraining, plus compliance review. API monthly cost equals volume times price per report.
Naming the equation first lets the interviewer follow, and challenge, any single term.
O
Own numbers.
60,000 reports a month, 35 cents per API call. Self-hosted: $150,000 upfront amortized over three years, four GPUs at $2.50 an hour, a 0.75 FTE ops engineer, quarterly retraining at $40,000, and $3,000 in compliance review.
Each figure traces to something checkable, a lease rate, a loaded salary fraction, an invoice, not an assumption pulled from the air.
U
Use a range.
Crossover lands between about 96,000 and 133,000 reports a month depending on the retraining cadence, quarterly versus twice a year.
A single crossover number implies more precision than the retraining assumption actually earns.
N
Nail the sanity check.
132,600 reports a month is a little over double FarSight's current volume, a believable multiple for a scale-dependent crossover, not an order-of-magnitude surprise.
Converting the crossover into a multiple of a trusted number is what catches a modeling mistake early.
D
Direction.
The retraining cadence moves the crossover more than any other assumption, since it's the single largest fixed cost line in the self-hosted total.
Knowing which assumption to pressure-test first is what separates an estimator from someone who just added five numbers together.
Try the same model somewhere elseSame five letters, a ticketing company's support-reply drafts instead of radiology reports. This time volume is high enough that self-hosting wins outright.
Mireille Kondo runs AI tooling at Osgard Ticketing, which drafts customer support replies for event refunds and transfers. Break it down the same way: self-hosted mostly fixed, API purely variable. Own numbers look different at Osgard's scale: 1.4 million support drafts a month, at 12 cents a report via API, that's 168,000 dollars a month. Self-hosted there needs more GPU capacity to handle the volume, about 22,000 a month, but the ops engineer, retraining, and compliance lines stay close to FarSight's, since one team can still run one retraining cadence regardless of throughput, near 26,500 a month fixed, plus the same amortized upfront near 4,200. Total self-hosted at Osgard's volume: about 52,700 a month. Use a range and nail the sanity check the same way: even doubling the GPU estimate to be conservative, self-hosted stays under a third of the API's cost. Direction: at this volume, the crossover was passed long ago, so the honest answer is self-host, and the real modeling question becomes how fast to migrate off the API contract, not whether to.
At FarSight's volume, calling a taxi for every ride is still cheaper than owning a van. At Osgard's volume, the van has been the better deal for a long time.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "self-hosted is mostly fixed cost, API scales with volume, find the crossover and compare it to your actual growth," and stop.
Cost: no time to get real GPU lease rates before the interview ends. Say so honestly, and use a stated placeholder rate, flagged clearly as a placeholder, rather than a false-precision number.
The model got better, for real: say a cheaper GPU generation drops lease costs by half. Rerun the crossover anyway, a cheaper self-hosted line just pulls the crossover volume down, it doesn't remove the need to check where it sits.
Where people run it wrong.
They compare only the per-report API price to the GPU lease rate, and skip the ops engineer and retraining lines entirely.
They treat self-hosting as cheaper on principle, without checking whether real volume clears the crossover.
They forget to amortize the upfront build cost, making self-hosting look artificially cheap in a spreadsheet nobody stress-tested.
How to use it live. The moment an interviewer asks to compare self-hosted and API costs, ask yourself first: which parts of this scale with volume, and which parts don't? That question alone buys real thinking time, and it's usually exactly where a real TCO model and a guess split apart.
One short aside about people
Estimation questions don't need a story. This one paragraph is worth adding anyway, because it's the part that makes the model wrong in practice if left out.
A cost model that says "self-host past 132,600 reports a month" assumes someone is actually watching the volume trendline against that number. In practice, the decision usually gets revisited only when a large invoice draws attention, not when the crossover is quietly approached. Tomasz Halvorsen, the radiologist whose reports make up a chunk of that volume, put it plainly when asked why nobody had flagged the growth: "I just write reports. Nobody ever showed me a number to watch." Worth one sentence in the interview: "I'd want a standing chart tracking volume against the crossover line, not a one-time model, because growth crosses that line quietly, long before anyone opens a new invoice."
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
Which framework fits comparing self-hosted and API total cost of ownership, and why not FLIPS?
Tap to flip
ANSWER
BOUND. There's no person's habit snapping here, just a fixed cost line against a variable one. Forcing FLIPS onto it would bolt a story onto a cost model that doesn't need one.
2 · THE EQUATION
State the self-hosted-versus-API comparison in one line.
Tap to flip
ANSWER
Self-hosted monthly cost equals amortized upfront plus GPU lease plus ops engineer plus retraining plus compliance, mostly fixed. API monthly cost equals volume times price per report, purely variable. Find where they cross.
3 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Bridget Osei, the AI PM modeling DraftScan's cost of ownership at FarSight Radiology. Tomasz Halvorsen is the radiologist whose report volume the whole comparison is modeled against.
4 · THE LOAD-BEARING NUMBER
Which assumption moves this crossover the most?
Tap to flip
ANSWER
The retraining cadence. Quarterly retraining sets the crossover near 132,600 reports a month. Twice-yearly retraining drops it to around 96,000, since retraining is the single largest fixed self-hosted cost.
5 · THE CROSSOVER
At what volume does self-hosting become cheaper than the API?
Tap to flip
ANSWER
Around 132,600 reports a month, a little over double FarSight's current 60,000-report volume.
6 · THE NUMBER
Fill in the blank: at 60,000 reports a month, self-hosted totals about ___ dollars, versus about ___ dollars for the API.
Tap to flip
ANSWER
$37,700 versus $21,000. The API is cheaper by nearly $17,000 a month at FarSight's current volume.
7 · THE SANITY CHECK
How do you check the crossover number in one step?
Tap to flip
ANSWER
Convert it into a multiple of current volume. 132,600 is a little over double FarSight's 60,000, a believable multiple for a scale-dependent crossover, not an order-of-magnitude surprise.
8 · CROSS PRODUCT TRANSFER
Section 4 runs BOUND again on a different product. Which one, and how does the outcome differ?
Tap to flip
ANSWER
Osgard Ticketing's support-reply drafts. At 1.4 million drafts a month, volume is already well past the crossover, so self-hosting wins outright and the real question becomes migration speed.
Check yourself Score: 0 / 0
Multiple choice
1. Why does the API stay cheaper than self-hosting at FarSight's current volume?
A. Vendor APIs are always cheaper than self-hosting, regardless of volume.
B. Self-hosting's cost is mostly fixed, GPU, ops, retraining, compliance, and 60,000 reports a month hasn't reached the volume where that fixed cost pays for itself.
C. FarSight's radiology reports are too complex for a self-hosted model to draft.
D. Self-hosted models are less accurate than the vendor's API.
Show hint
Look at the crossover line chart and where FarSight's current volume sits on it.
Show answer
B. Self-hosting's fixed costs need real volume to earn back, and FarSight's current 60,000 reports a month sits well below the roughly 132,600-report crossover.
True or false
2. True or false: the GPU lease is the largest single cost line in FarSight's self-hosted total.
True
False
Show hint
Look at the stacked-bar TCO build-up chart.
Show answer
False. Retraining, at $13,333 a month, is the largest single line, ahead of the ops engineer at $10,000 and the GPU lease at $7,200.
Fill in the blank
3. Fill in the blank: the self-hosted and API cost lines cross at approximately ___ reports a month.
Show hint
Look at the crossover marked on the line chart.
Show answer
132,600 reports a month. A little over double FarSight's current volume of 60,000 a month.
Short answer, where it wouldn't matter
4. Name a workload where this crossover logic would already favor self-hosting even below the calculated volume, and say why.
Show hint
Look at the quadrant diagram about where self-hosting actually wins.
Show answer
Model answer: A workload with very sensitive data and steady, predictable volume, where the value of keeping data in-house, and not paying for elasticity you never use, can justify self-hosting even before the pure cost crossover is reached.
Short answer, apply it yourself
5. Think of an API-based tool you use at real volume. Roughly estimate its fixed self-hosted cost versus its variable API cost. Where would the crossover sit?
Show hint
Think in terms of what would stay fixed regardless of volume, versus what scales directly with it.
Show answer
Model answer: A team using an API for email spam classification found their fixed self-hosted cost, mostly one part-time engineer and modest GPU needs, would cross their API's per-email cost only above several million emails a month, far beyond their real volume, so buying stayed the right call.
Short answer, work the number
6. If FarSight's volume grows 20 percent a year from 60,000 reports a month, roughly how many years until it reaches the 132,600 crossover?
Show hint
60,000 times 1.2 to a power, compared against 132,600.
Show answer
Model answer: About 4.5 years. 60,000 times 1.2 to the fourth power is roughly 124,400, and to the fifth power is roughly 149,300, so the crossover lands between year four and year five at that growth rate.
Before you close the answer
Why this works
Tests whether you can model a real crossover volume between a fixed-cost option and a variable-cost one, instead of declaring one option cheaper on principle or reciting cost categories with no arithmetic behind them.
Follow-up traps
"Couldn't a vendor offer volume discounts that change this?" Response: yes, and that's exactly why the API line should be modeled with the vendor's real tiered pricing if it exists, a flat per-report rate is a simplifying assumption worth naming and testing.
"Isn't data sensitivity more important than cost here?" Response: often yes, which is why the priority list ranks weighing sensitivity and control alongside cost, a close cost comparison can reasonably favor self-hosting for that reason alone, ahead of the pure crossover volume.
If pressed
The model assumes GPU cost scales smoothly with volume, but real GPU provisioning happens in discrete jumps, a fifth GPU added all at once, not fractionally, so the actual self-hosted cost line is a step function, not a smooth one, and the true crossover could land a little earlier or later than the smoothed estimate depending on exactly where those steps fall.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
