CalculationIntermediateQuality, Cost & Token Economics / Measuring ROI and business impact / #3
Model the annual ROI of a support agent that deflects 30 percent of tickets.
The direct answer
For a 500,000-ticket operation, somewhere between 0.5 and 2 million dollars a year net. But the number that decides whether any of it is real is how you define "deflected," not the 30 percent.
Do this, in order
- Pin down what "deflected" means before you multiply anything.Why: if it means "the bot answered and the user came back an hour later anyway," your real rate is half what you were told, and every number downstream is wrong.
- Write the equation out loud before you touch a number.Why: an interviewer is grading your structure. A wrong number inside a right equation is a small error. A right number with no equation is a guess.
- Say whether the saving is cash or capacity.Why: deflecting 30 percent saves nothing in cash unless headcount actually comes down. If nobody leaves, you bought capacity, which is real but it is not on the P and L.
- Subtract the run cost, not just the build cost.Why: inference is usually noise, but the people maintaining the knowledge base are not. That is the line most candidates forget entirely.
- Give a range, then sanity-check it against headcount.Why: converting the answer into "about 30 agents' worth" is the fastest way to catch an order-of-magnitude error before the interviewer does.
How to answer this, stage by stage
Six moves. This is an arithmetic question, so the words matter less than the structure, but the structure has to be said out loud.
1
Ask for the one number you can't invent
Say it like this
"Before I model this, one question: when we say deflected, do we mean the user never filed a ticket at all, or that the bot handled it first and some of them still came back? I'll assume the strict version and flag it, but it changes the answer by about half."
Why this works
This is the whole question hiding in one word. Asking it in ten seconds shows you've seen this metric lied about before. Then answer your own question and keep moving, don't wait to be rescued.
2
State the equation before any numbers
Say it like this
"So the shape is: annual ROI equals tickets per year, times deflection rate, times fully loaded cost per ticket, minus inference cost, minus the build and upkeep. Four terms in, two out."
Why this works
The interviewer can now follow everything you say next, and if they disagree with a number they can say so without derailing you. Nobody can follow a stream of numbers with no equation behind it.
3
Put a stake in each assumption, and say where it came from
Say it like this
"I'll say 500,000 tickets a year, which is a mid-size SaaS support org. Fully loaded cost per ticket: an agent costs about 60,000 a year and handles maybe 20 tickets a day over 230 days, so call it 13 dollars a ticket. I'd carry that as a range of 8 to 18 depending on complexity and where the team sits."
Why this works
"An agent costs 60k and handles 20 a day" is defensible. "About 13 dollars a ticket" on its own is a number you pulled from the air, and a good interviewer will ask where it came from.
4
Do the arithmetic slowly, and subtract the boring costs
Say it like this
"150,000 tickets deflected times 13 dollars is about 1.95 million gross. Inference is basically noise, call it 100,000 at 20 cents a conversation across everything it attempts. But two engineers and a PM to build and run it is around 500,000 loaded, and someone has to keep the knowledge base current, call that another 100,000. So net, about 1.25 million."
Why this works
Most candidates stop at the gross number. Naming the knowledge-base upkeep is the line that signals you've actually shipped one of these, because it's the cost nobody budgets for and everybody pays.
5
Sanity check against something you already know
Say it like this
"Quick check on that: 150,000 tickets at 4,600 tickets per agent per year is about 32 agents' worth of work. So this is roughly a 30-person question. That feels right for a 500k-ticket org, so I don't think I'm an order of magnitude off."
Why this works
Converting money back into people catches the errors that matter. If it had come out as 3 agents or 300, you'd know instantly something was wrong, and you'd have caught it before the interviewer did.
6
Name the assumption that would break it
Say it like this
"The number I'd want to pressure-test isn't the cost per ticket, it's the deflection definition. If 40 percent of deflected users file anyway, real deflection is 18 percent, not 30, and the whole thing drops to around 700,000. And separately: if nobody's headcount actually comes down, that 1.25 million is capacity, not cash. Worth saying which one the CFO is expecting."
Why this works
Naming the assumption that moves the answer most is what separates an estimator from a calculator. The cash-versus-capacity distinction is the one that gets ROI models killed in review, and almost nobody raises it unprompted.
If you remember one thing
The interviewer is not checking your arithmetic. They're checking whether you know which number is load-bearing. Here it's the definition of "deflected," and it's worth more than every other assumption combined.
Let's learn
Here is the whole model in one block, so you can see the shape of it before we argue about any single number.
# inputs
tickets_per_year = 500,000 # mid-size SaaS support org
deflection_rate = 30% # the number we were handed
cost_per_ticket = $13 # $60k agent / 4,600 tickets a year
# gross saving
500,000 x 30% x $13 = $1,950,000
# costs out
inference = -$100,000 # 500k attempts x ~$0.20
build + run (2 eng, PM) = -$500,000
knowledge base upkeep = -$100,000 # the line nobody budgets
# net
= $1,250,000 # range: $0.5M to $2.0M
tickets_per_year = 500,000 # mid-size SaaS support org
deflection_rate = 30% # the number we were handed
cost_per_ticket = $13 # $60k agent / 4,600 tickets a year
# gross saving
500,000 x 30% x $13 = $1,950,000
# costs out
inference = -$100,000 # 500k attempts x ~$0.20
build + run (2 eng, PM) = -$500,000
knowledge base upkeep = -$100,000 # the line nobody budgets
# net
= $1,250,000 # range: $0.5M to $2.0M
Where the money goes
Gross saving
+$1.95M
Inference
-$0.10M
Build and run
-$0.50M
Knowledge base upkeep
-$0.10M
Net
$1.25M
Inference is the line everyone expects to dominate and it's the smallest bar on the chart. The people costs are six times larger.
Knowledge spark: fully loaded cost
"Fully loaded" means salary plus everything else it takes to keep that person working: benefits, payroll tax, a manager's time, a seat, the software licences. It usually runs 1.25 to 1.4 times base salary. If you model with base salary alone, you will understate every people-cost line in the sheet.
The trap in the word "deflected"
The same 30 percent, two very different numbers
Both of these get counted as a deflection in most dashboards. Only the first one saves you anything.
If the bot answers, the user goes away unhappy, and forty minutes later they file a ticket anyway, you have not deflected that ticket. You have added a step to it. The agent who eventually picks it up now has a slightly annoyed customer and a transcript to read first.
A deflection rate measured at the moment the bot replies is measuring the bot's confidence, not the user's satisfaction.
So the number to ask for is deflection measured at 24 or 48 hours: of the conversations the bot closed, how many produced no human ticket from that same user in the next two days. That number is always lower, and it is the only one worth putting in a model.
At 40 percent leak-back, the headline 30 percent becomes a real 18 percent, gross saving falls from 1.95 million to 1.17 million, and net drops to roughly 470,000. Same product, same model, same everything. One definition.
BOUND, in one screen
This is an estimation question, so the framework is BOUND. There is no story here and no person to follow. FLIPS deliberately does not fit this shape, and forcing it would produce a parable with arithmetic bolted to the side.
BOUND, for estimation and sizing questions
B, break it down. Tickets times deflection times cost per ticket, minus inference, minus build and upkeep.
O, own numbers. 500k tickets. $60k agent handling 4,600 a year, so $13 a ticket. Each one traceable to something a person could check.
U, use a range. $0.5M to $2.0M net, driven mostly by cost per ticket ($8 to $18) and by the deflection definition.
N, nail the sanity check. 32 agents' worth of work. Right order of magnitude for a 500k-ticket org.
D, direction. The deflection definition moves the answer more than every other assumption combined. Cash versus capacity is the second one.
One short aside about people
Estimation questions do not need a story. This one paragraph is worth adding at the end anyway, because it is the thing that makes the model wrong in practice.
Deflection models assume the tickets that go away are a random 30 percent. They are not. The bot deflects the easy ones: password resets, where is my invoice, how do I change my plan. What is left for the humans is the hard, slow, angry residue. So average handle time on the remaining 70 percent goes up, and the agents who used to get a breather between two difficult calls now get another difficult call. Your cost per remaining ticket rises, your team's attrition rises, and neither shows up anywhere in the model above. Worth one sentence in the interview: "I'd want to watch handle time on the tickets that don't get deflected, because the mix is getting harder, not just smaller."
Flashcards (click a card to flip it)
1 · THE FRAMEWORK
Which framework fits an estimation question, and why not FLIPS?
Tap to flip
ANSWER
BOUND. FLIPS finds a behaviour that snaps; there is no behaviour here, just arithmetic. Forcing it produces a story with numbers bolted on.
2 · THE EQUATION
State the ROI equation in one line.
Tap to flip
ANSWER
Tickets x deflection rate x fully loaded cost per ticket, minus inference, minus build and upkeep.
3 · THE LOAD-BEARING NUMBER
Which assumption moves this answer the most?
Tap to flip
ANSWER
The definition of "deflected." At 40 percent leak-back, net drops from about $1.25M to about $470k. Nothing else comes close.
4 · CASH OR CAPACITY
When does a deflection saving show up on the P and L?
Tap to flip
ANSWER
Only if headcount actually comes down. Otherwise you bought capacity, which is real but is not cash. Say which one the CFO expects.
5 · THE FORGOTTEN COST
Which cost line do candidates almost always miss?
Tap to flip
ANSWER
Knowledge base upkeep. Someone has to keep the content current or deflection decays. Inference, which everyone worries about, is the smallest line.
6 · THE SANITY CHECK
How do you check an ROI number in one step?
Tap to flip
ANSWER
Convert it back into people. 150,000 tickets at 4,600 per agent is about 32 agents. If it came out at 3 or 300, something is wrong by an order of magnitude.
7 · FULLY LOADED
What does "fully loaded cost" include, and what happens if you skip it?
Tap to flip
ANSWER
Salary plus benefits, payroll tax, management time, seat, licences. Usually 1.25 to 1.4x base. Skip it and you understate every people-cost line.
8 · THE MIX SHIFT
Why does average handle time go up after you deploy a deflection bot?
Tap to flip
ANSWER
The bot takes the easy tickets. What's left for humans is the hard, slow residue. The remaining 70 percent is harder, not just smaller, and no ROI model shows it.
Check yourself Score: 0 / 0
Fill in the blank
1. Gross saving equals tickets per year, times ______, times ______.
Show hint
Two of the three input terms from the equation.
Show answer
Deflection rate, and fully loaded cost per ticket. 500,000 x 30% x $13 = $1.95M gross, before any costs come out.
True or false
2. True or false: inference cost is usually the largest expense line in a deflection ROI model.
Show hint
Look at the chart. Which bar is smallest?
Show answer
False. Inference came to about $100k against $600k of people cost. It's the line everyone expects to dominate and it's the smallest one on the chart.
Multiple choice
3. A dashboard reports 30 percent deflection. What's the first thing you ask?
Show hint
Which answer could cut the ROI in half?
Show answer
B. Deflection measured at the moment the bot replies is measuring the bot's confidence, not the user's satisfaction. A 48-hour window is the only version worth modelling, and it's always lower.
Multiple choice
4. Your model says $1.25M saved. The CFO says the budget didn't change. Who's wrong?
Show hint
What has to physically happen for a headcount saving to hit the books?
Show answer
B. Deflecting 30 percent of tickets frees 32 agents' worth of time. If none of those roles go away or go unfilled, the money never leaves the cost base. Both parties are right, they're measuring different things, and saying so out loud is the whole point.
Short answer
5. Redo the model with 200,000 tickets a year and a $9 fully loaded cost per ticket. What's the gross saving, and does the project still make sense?
Show hint
Run the same three input terms, then compare the result to the fixed costs, which do not shrink with volume.
Show answer
Model answer: 200,000 x 30% x $9 = $540,000 gross. But build and upkeep is still about $600,000, and inference maybe $40,000. So it's roughly break-even or slightly negative in year one. The lesson: fixed costs don't scale down with volume, so deflection ROI is strongly a function of ticket volume. Below a few hundred thousand tickets, buy rather than build.
Short answer, apply it yourself
6. Pick any automation you've seen proposed at work. Write its equation in one line, then name the single assumption that would move the answer most.
Show hint
The load-bearing assumption is usually a definition, not a number. "Deflected," "active user," "resolved," "qualified lead."
Show answer
Model answer: "Invoice auto-matching: invoices per month x auto-match rate x minutes saved per invoice x loaded hourly rate, minus build and run. The load-bearing assumption is what counts as auto-matched. If a match still gets eyeballed by a human before posting, the minutes saved is 30 seconds, not 4 minutes, and the whole case collapses." Any answer works if the load-bearing item is a definition rather than a number.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
- A live AI agent you actually shipped
- A launch decision you can defend under pressure
- An interview-ready portfolio, not more flashcards
More on Measuring ROI and business impact
- #1 How do you build the ROI case for an AI feature before it ships?
- #2 What is the difference between time saved and value created?
- #4 How do you attribute a revenue change to an AI feature specifically?
- #5 Explain why time-saved metrics are frequently overstated.
- #6 Describe an experiment design that would isolate an AI feature's business impact.
- #7 What ROI argument works for an internal AI tool with no revenue line?