The direct answer
Because the mean tells you about requests and the p95 tells you about people. An average of 780 milliseconds can hide 3,000 couriers a night sitting on a frozen screen, and those aren't a random 3,000. They're the ones working the busiest zone at the busiest hour.
Do this, in order
Set the target on the p95, not the mean.Why: an average blends the slow calls into the fast ones until they vanish. The p95 is the only number that still has the slow people in it.
Start and stop the clock on the thing the user is staring at.Why: a p95 of 300ms on one internal call is worthless if the accept screen still takes four seconds. You will hit the number and lose the courier.
Count timeouts and errors as slow, at the timeout value. Never drop them.Why: a call that never returns has no time to record, so it quietly leaves the sample. Do that and your p95 improves as the system gets worse.
Cut the p95 by zone and by hour. Never average p95s together.Why: one fleet-wide p95 is the mean sneaking back in through a side door. The whole point was that the slow calls are clustered, so keep the clusters visible.
Make p95 the target and treat p99 as a bug list.Why: p99 here is 400 calls a night, and it's usually one retry loop or one cold start. That's worth a ticket. It is not worth a team's quarter.
Write down what you do at each number before you publish it.Why: a target with no action attached is a wall decoration. Tomás watched a green board for four months and it never once told him to do anything.
How to answer this, stage by stage
Five moves. This one rewards someone who does the arithmetic out loud, so lead with numbers and let the judgement follow them.
1
Start with the arithmetic, out loud
Say it like this
"Let me put numbers on it, or this is just a preference. Take one evening of a food-delivery dispatch service: 40,000 couriers tapping accept on an order. 37,000 of those come back in about a fifth of a second. 3,000 take between six and fifteen. The average of all that is about 780 milliseconds, which is green against a one-second target. The p95 is seven seconds."
Why this works
It settles the question in ten seconds without an argument. Once both numbers are on the table, the interviewer can see the gap themselves, and everything you say next is explaining a gap rather than defending an opinion.
2
Say who the slow calls actually are
Say it like this
"And here's the part that decides it. That slow 3,000 isn't a random 3,000. Seven out of ten of them land between seven and nine at night, in the three busiest zones, because that's exactly when the most orders are open and the fewest couriers are free. So the tail doesn't sprinkle a bit of pain over everyone. It sits down hard on the busiest hour of the week."
Why this works
Most candidates stop at "the mean hides the tail," which is true and slightly boring. This is the line that shows you know why the tail exists: slow requests are slow because the work was hard, and the hard work is where the money is.
3
Commit to p95, and say why not p99
Say it like this
"So I'd set it at p95. Not the mean, and not p99. p95 is 2,000 calls a night here, which is a real group of people you could go and stand next to. p99 is 400, and every p99 I've dug into turned out to be one thing: a retry loop, or a cold start right after a deploy. I'd file that as a bug. I wouldn't hang a team's quarter on it, because you can chase a p99 forever."
Why this works
Picking a number and defending its boundary on both sides is the whole skill being tested. "Use percentiles" is a slogan. "p95 and here's why not p99" is a decision someone can argue with.
4
Set the clock on what the courier sees
Say it like this
"One thing I'd pin down before I agree to any number: what starts and stops the clock. If we measure the courier-scoring call on its own, we can hit 300 milliseconds and the accept screen still takes four seconds, because there's a payment hold and a merchant ping after it. So the target goes on the whole thing he's staring at, from tap to confirmation."
Why this works
This is the failure that actually happens in real teams. The service owner hits their number, the product gets slower, and nobody is lying. Naming it shows you have shipped something, not just read about percentiles.
5
Attach an action to every number
Say it like this
"And I'd write down what we do at each number, or it's just a dashboard. Under a second, leave it alone. One to three seconds, put a real waiting state on the screen and cap retries at one, because right now people double-tap and we double-assign the order. Over three seconds two nights running in any single zone, we stop opening new zones until it's fixed."
Why this works
A metric nobody acts on is decoration. Three thresholds with three named actions is the difference between a candidate who has watched a dashboard and one who has owned it.
If you remember one thing
Stages 1 and 2 are the answer. Do the arithmetic, then say who the slow requests belong to. If you only have thirty seconds, say those two things and stop talking.
Let's learn
Every evening, about 40,000 couriers tap accept on an order in one city. Each tap asks the dispatch service a question: is this order mine now? The screen holds still until it answers.
On the wall of the dispatch room there was one number for that, refreshed every minute. Average dispatch time. It sat around 780 milliseconds and the target was one second, so the number was green. It was green in March, it was green in April, and it was green on the worst Friday of the year.
Median
170ms
what a typical courier waits
Mean
780ms
the number on the wall
p95
7 sec
2,000 calls a night are worse than this
p99
13 sec
400 calls a night
Look at those four numbers for a second, because they all describe the same 40,000 taps. Not four systems. One system, four questions.
40,000 dispatch calls, one evening, drawn at true count
median 170ms
mean 780ms
p95 about 7s
<100ms
100-200
200-300
300ms-1s
1-3s
3-6s
6-9s
9-15s
The three grey bars are 37,000 calls, 92.5 percent of the evening. The five rust bars are 3,000 calls, and at true scale you can barely see four of them. That is the whole problem: the thing that ruins your night is invisible on the chart and invisible in the average. Note the lump on the far right, 1,450 calls sitting at 9 to 15 seconds. Slow calls pile up at a retry boundary, they do not spread out smoothly.
Knowledge spark: what a percentile is
Line every call up, fastest on the left, slowest on the right. The p95 is the one standing 95 percent of the way along. Nineteen out of twenty were faster than it. One in twenty was slower. The mean is a different move: add every call up and divide by how many there were. One nine-second call drags the mean up for everybody. It barely nudges the p95, because the p95 only cares about position in the line, not size.
Now put the mean and the median side by side. The mean is 780ms. The median is 170ms. They are four and a half times apart, and that gap is the whole story: it means the fast calls and the slow calls are not one crowd with a middle. They are two crowds with a canyon between them. Almost nobody actually waits 780 milliseconds. That number sits in the empty space where nothing happens.
And the slow crowd has a shape. Of those 3,000 slow calls, 2,130 land between seven and nine at night. That is 71 percent of the pain in 33 percent of the shift. Around 2,400 of them come from three of the eleven zones: the centre, the university, the stadium end. Those are the zones with the most open orders and the fewest free couriers, which is exactly why the scoring service has the longest list to sort and takes the longest to answer.
The mean tells you about requests. The p95 tells you about people. And the slow requests are not spread evenly across people.
That is why "5 percent" is a misleading way to describe it. It sounds like a rounding error you could accept. It is not a random 5 percent skimmed off the top. It is the busiest hour, in the busiest zone, on the orders worth the most, hitting the couriers who work the hardest shift. You built a system that gets slowest exactly when it matters most, and then you measured it with a number designed to average that away.
Same 40,000 calls, two different questions
The decision I would take back
We put the fleet average on the wall board and hung the on-call alert on it. Replace both with the p95 per zone, and let the alert fire on any single zone going red, not on the fleet. Same screen, same one line of code in the alert rule, completely different behaviour in the room.
Why that was a sensible decision at the time. One number is easy to read across a room. An alert needs a single threshold or it never gets set up at all. And when the service launched there were two zones, both quiet, and the mean and the p95 were 40 milliseconds apart. The decision was right on the day it was made and it aged badly, which is the normal way these things go.
What I would leave alone. The end-of-shift earnings screen. A courier opens it once, at eleven at night, sitting down. If it takes four seconds, nothing breaks, nobody double-taps, no order gets double-assigned. Putting a p95 target on that screen would cost real engineering time to protect a moment where nobody is waiting on a decision. Percentile targets belong where a person is blocked, not everywhere a request exists.
Knowledge spark: why you cannot average percentiles
If the centre zone has a p95 of 12 seconds and a quiet suburb has a p95 of 400 milliseconds, the two together do not make a zone at 6.2 seconds. Nobody experienced 6.2 seconds. To get the real combined p95 you have to go back to the raw calls and line all of them up again. Any dashboard that shows you an average of percentiles has put the mean back in without telling you.
The lesson. I used to think picking p95 over the mean was a statistics preference, the kind of thing you defend with a textbook. It is not. It is a decision about whose evening you are willing to not see. Every summary number you put on a wall is a choice about which people get averaged into the background, and the mean is the setting that hides the most people. If you only ever look at one number, at least look at the one that still has a person in it.
The evening Tomás stopped trusting the board
You do not need this to answer the question. Read it when you want to feel why the number matters, rather than just knowing which one to pick.
Tomás has run the evening dispatch desk for eleven years, four of them before there was an app at all. Back then it was a radio and a whiteboard with eleven columns on it, and he was very good. He could look at the board at half past six and tell you which zone was about to go bad, twenty minutes before it went bad. Nobody taught him that. He just knew what a thin column looked like on a wet Friday.
When the dispatch service arrived it took most of that off him, and he was glad. Assignment stopped being his job and became a thing the system did. What he kept was the map: eleven zones on a screen, and the wall board above it with the average dispatch time, green, one line.
For four months it was green and it was fine. He liked the board. It meant he could spend the dinner rush watching the map instead of watching the plumbing, and the map was the part he was actually good at.
The courier is waiting for the last box. We were measuring the second one.
Then, on a Tuesday in October, a courier called Priya came up to the counter while she was waiting on a pickup. She was not angry. She was not complaining. She asked him one question, in the tone you use for a thing you assume is your own fault: "Does the accept button work properly in the centre?"
Tomás looked up at the board. 810 milliseconds. Green. He said he would look into it.
Here is what he did next, and it is completely reasonable. He started standing at the pickup counter between seven and nine, watching couriers' phones over their shoulders while they tapped accept. Two hours a night, doing by eye what the wall board was supposed to be doing for him. He found it in about a week. On a busy zone at half past seven, the screen would sit there for six, eight, eleven seconds, and the courier would tap again, and sometimes the second tap took a second order.
The board was never lying. It was averaging Priya's nine seconds against nineteen people who were already gone.
The two hours he spent at the counter were the two hours he used to spend on the map. So we did not just miss 3,000 slow calls a night. We took the one person in the building who could see a zone going bad twenty minutes early and put him at a counter, counting to nine.
And nobody ever raised a ticket. Priya asked once and then stopped asking, because she had been told it was fine and the board said fine. The couriers who waited nine seconds mostly assumed it was their own phone. That is what a bad tail looks like from the inside: not a complaint, just people quietly deciding the problem is them.
Now replay it with p95 per zone on that wall instead. It is half past seven on the same Tuesday. The centre zone reads 9.4 seconds and it is red. Tomás does not need Priya, and he does not need a week at the counter. He calls it in at 19:34, the dispatch team finds the retry loop that night, and he is back on the map for the rest of the rush.
Same board. Same screen. Same one line of code. The only thing that changed is which question the number was answering.
LEAD, one letter at a time
This is a metric question, so the framework is LEAD. A "what if dispatch got twice as slow" question would use FLIPS instead. Different question shapes, different tools.
LEAD, for metric questions
L, link. The outcome that pays the bills: orders delivered on time, and couriers still logged on at nine. Not "dispatch latency." Nobody buys dinner because a service was fast.
E, early signal. p95 accept time, split by zone and by hour. It goes bad weeks before a courier logs off for good, and months before anyone writes it in a survey. This is the answer to the question.
A, abuse. Four ways this number gets hit without doing the work, and all four have happened somewhere. Drop the timeouts from the sample, so calls that never returned stop counting. Measure the scoring call instead of the accept flow. Average the p95 across zones, which is the mean wearing a disguise. Count one retried call as two fast calls instead of one slow one.
D, decision. Under one second, do nothing. One to three seconds, ship a real waiting state and cap retries at one. Over three seconds two nights running in any zone, stop opening new zones until it is fixed.
The D step, drawn out
The test for LEAD
Name the metric that would look perfectly healthy right up to the morning everything broke, then say why. Here it is the mean, and the why is that 92.5 percent of the calls are fast enough to outvote the other 7.5 percent no matter how bad they get. If you cannot name that metric, you are still measuring the outcome instead of the leading edge.
Run LEAD on a warehouse floor
A picking app in a grocery distribution centre. A picker scans a bin, and the app tells them which shelf to walk to next. Different building, different job, same shape of mistake waiting to be made.
L, link. Orders packed per shift, and how many pickers finish before their shift ends.
E, early signal. p95 time from the scan to the next shelf appearing, split by aisle and by hour of the shift.
A, abuse. The vendor reports one p95 for the whole warehouse. The frozen aisle is 4 percent of picks, it has the worst wifi in the building, and its own p95 is eleven seconds. In a warehouse-wide number it disappears completely, and the pickers on frozen rotation are the ones who quit.
D, decision. Under 800 milliseconds, leave it. Over two seconds in any aisle, load the next three shelves onto the device before the picker gets there, so the wifi stops being in the loop at all.
Swap the trigger and it still runs
- It gets faster: the median drops from 170ms to 90ms. Every average on every dashboard improves, and not one courier's evening is any different, because the 3,000 slow calls did not move.
- It gets more expensive: you move to a cheaper region and the median goes up 40 milliseconds. Nobody can feel 40 milliseconds. The p95 is the only number worth arguing about in that meeting.
- It gets better: the team fixes the retry pile, so those 1,450 worst calls drop from 9 to 15 seconds down to under a second. The mean goes from 780ms to 370ms, which reads as "we saved 0.4 seconds." The p95 goes from 7 seconds to about 0.7. Same fix. The mean undersells it by about fifteen times, and the biggest win the team shipped all year looks like a rounding error on the wall board.
Where people run it wrong
- Setting the p95 on the model call or the database call, hitting it, and celebrating while the screen the user is looking at still takes four seconds.
- Averaging the p95s. Two zones at 400 milliseconds and 12 seconds do not make a zone at 6.2 seconds. You have to go back to the raw calls.
- Setting p95, p99 and p99.9 targets all at once. Now nobody knows which one to wake up for, and the team ends up chasing whichever one is red today.
If you are asked this cold
Buy yourself ten seconds by asking what the clock is measuring. "Before I pick a number, let me say what we are timing, because that decides the rest. Is this a person watching a spinner, or a background job nobody sees?" That is not stalling, it is stage 4 pulled to the front, and it gives you the time to line up the arithmetic in your head.
Flashcards (click a card to flip it)
1 · THE FRAMEWORK
Which framework fits a metric question, and why not FLIPS?
Tap to flip
ANSWER
LEAD: link, early signal, abuse, decision. FLIPS is for "what if X changed," where you hunt a behaviour that snaps. Here the job is picking which number to watch, so LEAD's E step is the answer.
2 · THE PERSON
Who is this answer about, and what was he good at?
Tap to flip
ANSWER
Tomás, evening dispatch supervisor, eleven years in. He could look at the zone map at half past six and name the zone about to go bad twenty minutes early.
3 · THE FOUR NUMBERS
Median, mean, p95, p99 for one evening. What are they, and why do they disagree?
Tap to flip
ANSWER
170ms, 780ms, 7 seconds, 13 seconds. Same 40,000 calls. They disagree because the calls sit in two crowds with a canyon between them, and each number picks a different crowd.
4 · THE CORRELATION
Why does it matter that the slow 7.5 percent is not random?
Tap to flip
ANSWER
71 percent of it lands between 7 and 9pm, mostly in three of eleven zones. Slow calls are slow because the work is hard, and the hard work is the busiest hour with the most orders open. The tail lands on your best moments.
5 · THE OLD DECISION
What decision does this answer take back, and why was it sensible?
Tap to flip
ANSWER
Putting the fleet average on the wall board and hanging the alert on it. Sensible at launch: two quiet zones, mean and p95 40ms apart, and an alert needs one threshold. It aged badly.
6 · THE WORST ABUSE
Name the nastiest way a p95 gets gamed.
Tap to flip
ANSWER
Dropping timed-out calls from the sample. A call that never returns has no time to record, so it silently leaves. The worse the system gets, the better the number looks. Count them at the timeout value.
7 · P95 OR P99
Why stop at p95 instead of going to p99?
Tap to flip
ANSWER
p95 is 2,000 calls a night, a real group of people. p99 is 400, and it is usually one retry loop or one cold start after a deploy. That is a bug to file, not a target to chase, and a p99 can be chased forever.
8 · TRANSFER
The last section runs LEAD on a different product. Which one, and what is its early signal?
Tap to flip
ANSWER
A warehouse picking app. Early signal: p95 time from bin scan to next shelf appearing, split by aisle and hour, because the frozen aisle (4 percent of picks, 11-second p95) vanishes inside a warehouse-wide number.
Check yourself Score: 0 / 0
Multiple choice
1. Which of these would make your p95 look better while the system actually got worse?
- A. Making the fastest 50 percent of calls twice as fast.
- B. Leaving timed-out calls out of the sample because they have no recorded time.
- C. Reporting the p95 hourly instead of daily.
- D. Adding more couriers to the busy zones.
Show hint
Which one removes the very slowest calls from the line you are counting along?
Show answer
B. A call that never returns has no duration, so it quietly falls out of the sample. Every extra timeout makes the surviving calls look faster. Option A does nothing to the p95 at all, because speeding up calls that are already on the fast side of the line does not move where the line's 95 percent mark falls. Count timeouts at the timeout value.
Fill in the blank
2. You cannot ______ percentiles. A zone at 400ms and a zone at 12 seconds do not combine into a zone at ______.
Show hint
The second blank is a number nobody in either zone actually experienced.
Show answer
Average. 6.2 seconds. To get the real combined p95 you have to line up the raw calls from both zones and find the 95 percent mark again. A dashboard showing an average of percentiles has put the mean back in without telling you, which is the exact thing you were trying to escape.
True or false
3. True or false: if the mean dispatch time is 780 milliseconds, then a typical courier waits about 780 milliseconds.
Show hint
Compare the mean to the median on the four-number strip. What is standing between them on the chart?
Show answer
False. The median is 170ms, so a typical courier waits about a fifth of a second. The mean is 780ms because 3,000 calls in the 6 to 15 second range drag it up. Almost nobody waits 780ms. That number lands in the empty gap between the two crowds, and it describes nobody's evening.
Multiple choice
4. The team fixes the retry pile. The mean drops from 780ms to 370ms. The p95 drops from 7 seconds to about 0.7 seconds. Which number goes in the launch write-up?
- A. The mean, because it covers every call rather than a slice.
- B. The p95, because it says what changed for a person, and the mean undersells the same fix by about fifteen times.
- C. Both, weighted together into a single score.
- D. Neither. Report the number of calls fixed instead.
Show hint
One version reads "we saved 0.4 seconds." The other reads "we cut a seven-second wait to under one."
Show answer
B. The mean hides your wins for exactly the same reason it hides your failures: 37,000 fast calls outvote everything. Reported as a mean, the biggest fix of the year looks like 0.4 seconds. Reported as a p95, it is 7 seconds down to 0.7, which is the thing the courier actually felt.
Short answer
5. Suppose the team only half-fixes the retry pile: 725 of those 1,450 worst calls now come back in under a second, and the other 725 do not change. Roughly what happens to the mean, and roughly what happens to the p95?
Show hint
For the mean, swap 725 calls from about 12 seconds to about 0.65 seconds and re-divide by 40,000. For the p95, count along the line to the 38,000th call and see which bucket it now lands in.
Show answer
Model answer: The mean goes from about 780ms to about 570ms, a drop of roughly a quarter. The p95 goes from about 7 seconds to about 1.8 seconds, a drop of roughly three quarters. Same half-fix. The mean shrugs and the p95 reacts hard, because moving 725 calls barely shifts an average over 40,000 but it moves the 38,000th position in the line by two whole buckets. This is the same lesson as question 4, one size down.
Short answer, apply it yourself
6. Pick an app you use most days. Name one moment in it where the average speed would look perfectly fine while one in twenty of your own uses was awful, and say who that slow one in twenty would be.
Show hint
Look for a moment where the app has to do more work when your situation is harder: a bigger file, a worse signal, a longer list, a busier time of day. That is where a tail comes from.
Show answer
Model answer: "Mobile banking, opening the transactions list. Nineteen times out of twenty it is instant, because I have forty transactions. On the first of the month, right after payday, when everyone opens it at once and my list is longest, it can hang for eight seconds and I tap it again. The slow one in twenty is not random: it is people with lots of transactions checking on the busiest day, which is the people the bank most wants inside the app." Any answer works as long as the slow slice is explained by something that makes the work harder, not by luck. If your example's slow cases really are random, that is a fine finding too, and it means a mean would have been safe there.
