What acceptance criteria would you set for latency at the 95th percentile and why not the mean?

How to answer this, stage by stage

Six moves. This is arithmetic with a single honest split hiding inside it, so what's being graded is whether you find the split before the interviewer has to point at it.

Let's learn

Say Ondo Mobile, a telecom company, builds an AI that answers its customer support line. No hold music, no queue. It listens to what the caller wants and tries to solve it on the spot, and hands off to a person only when it has to.

Before the AI, every one of those 26,000 daily calls sat in a hold queue first, about three minutes on average, before a human agent even picked up. Now the AI answers instantly. The pause between the caller finishing a sentence and the AI starting to reply averages 900 milliseconds. On the launch dashboard, that number was green.

Here's the part that matters. The average being fine was never the problem. The problem is what a caller does the moment their own pause runs long. They say "hello, are you still there?" Or they just hang up.

The equation behind that pause is short. It equals the time to notice the caller's stopped talking, plus the time the AI takes to decide what to say, plus, only for calls that need it, a call out to the billing or account system, plus the time it takes the reply to start being spoken.

Here's what that cost at its worst. A caller who hangs up mid-pause doesn't just lose four seconds. They land back in the exact hold queue the AI was built to replace, except now they've also burned the call they already made. The product ends up slower than if we'd sent them straight to a person.

The choice I would take back. Writing the acceptance criteria as one blended number, mean response time under 1 second, for every call, instead of splitting it by what the call actually needs. That was fine when almost no calls touched the billing system. It stopped being fine the week bills went out.

What I would leave alone. The text summary we email after the call ends. Nobody is sitting on the line waiting for that one. It could take ten seconds and not a single caller would notice.

The lesson. If a single average can hide the exact calls where a caller decides the product is broken, the acceptance criteria was never really finished. It measured the easy nine calls out of ten and called that the whole line.

Now here is the same thing as a story

Skip this if a caller's dead-air pause already keeps you up at night. Read on if it doesn't yet.

Ines Petrescu has run product for Ondo Mobile's support line for three years. She wrote the acceptance spec for the AI voice line herself, the same week it went into pilot.

For the first four months, her number held up fine. About eight calls in a hundred needed the billing lookup, most of them people asking about a roaming charge. The other ninety-two never left Ondo's own systems: check a balance, reset a PIN, confirm a SIM is active. Mean response time sat at 900 milliseconds. The launch dashboard was green, and green was all anyone checked.

Then the billing cycle caught up with the AI the way it always used to catch up with the old queue, except nobody had planned for it this time. Three days after invoices went out each month, the complex-call share didn't creep, it jumped, eight percent to something closer to forty. People calling about a charge they didn't recognize, wanting their account pulled up, sometimes asking to be checked for fraud. Each of those calls now had to reach into the billing platform, and the billing platform, under that same monthly spike, started timing out and retrying.

Nobody noticed right away, because the mean barely moved. 900 milliseconds crept to 1.05 seconds, which read as a rounding error against the under-a-second target, not a breach anyone would escalate.

It surfaced because Baraka, who runs QA on the voice line, pulled five hundred recorded calls from that billing week, the same way he pulled a sample every quarter. He wasn't looking for anything specific. He was listening for calls that sounded odd.

Sixty one of the five hundred had a caller saying something like "hello, are you still there?" partway through, or just hanging up mid-pause. Fifty eight of those sixty one were on the billing-lookup path. He played Ines three of the recordings in her office on a Wednesday, without much comment. He didn't have to say much.

She remembered writing the spec. It was the same week the pilot launched, and the number she wrote was "mean response time under 1 second." It was true, and it was easy to put on one line of a slide. Splitting it by call type would have meant admitting the billing lookup was slower and less certain than the rest of the flow, and at eight percent of calls, that felt like a footnote, not a target.

The choice I would take back. One blended average, written the same confident way for every call, instead of two numbers with the billing lookup called out on its own line.

What I would leave alone. The post-call text summary. It's not real time, nobody's holding the phone waiting on it, and it could take ten seconds without one caller noticing.

The spec she rewrote after that Wednesday gives the billing lookup its own p95 line, 2.5 seconds, next to 1.5 seconds for everything that stays inside Ondo's own systems, both checked every week against the real mix instead of whatever the mix happened to be at launch.

Same billing week, same call volume, new spec: the complex-call p95 target holds at 2.5 seconds instead of measuring 4.2. Baraka's next quarterly sample, five hundred calls again, turns up six dead-air calls, not sixty one.

The part I'd go back and tell myself: I asked how fast the average call was. I never asked how many different kinds of call I was averaging together.

Same voice line. The fix was never a faster call, it was a second line on the spec.

BOUND, letter by letter, for Ondo Mobile's phone line

This is a sizing question with an honesty problem riding on top, so BOUND fits and FLIPS doesn't. Nobody's behavior snapped here. A PM wrote a number that felt safe and it was really an average of two very different calls.

The range against how a phone call actually feels. Billing week landed past every mark on the line.

B, break it down. The pause a caller feels equals the time to notice they've stopped talking, plus the time to decide what to say back, plus, only for calls that need it, a call out to the billing or account system, plus the time it takes the reply to start being spoken.
O, own the numbers. 300 milliseconds to notice the caller's stopped, from our own voice pipeline's telemetry. 450 to 500 milliseconds to decide what to say, from the routing service's own dashboard. 900 milliseconds for a typical billing lookup, borrowed from the account platform team, and a number Ines had never seen under real load. 250 milliseconds for the reply to start speaking, from the voice vendor's own numbers.
U, use a range. About 1 second typical for a call that never leaves our own systems, p95 target 1.5 seconds. About 1.95 seconds typical for a call that needs the billing lookup, p95 target 2.5 seconds. Blended across a normal week, roughly 1.7 seconds. Blended during billing week, measured, 4.2 seconds.
N, nail the sanity check. Even the 1.5 to 2.5 second target is already five to twelve times longer than a normal conversational pause, a ceiling to defend, not a cushion. Against the old three-minute hold queue, even the worst measured pause is about forty times shorter. The tool was never the risk. Blending two very different calls into one number was.
D, direction. Traffic mix swings the target the most. At the normal one-in-ten complex share, the blended p95 stays close to 1.7 seconds even with a slow lookup. At four in ten, it blows past 4 seconds even if every system runs exactly at spec, because more calls are simply sitting in the slow lane. A second, weaker lever: how often the lookup times out and retries. One retry adds about 1.5 seconds to that single call, and raising the retry rate from 1 in 200 complex calls to 1 in 20 roughly doubles the complex-only p95 on its own. Mix is the bigger lever, worth naming both if asked why not the other one.

And if you want to be sure it really works, try it somewhere else

Rivermark Water, a municipal utility, runs a similar line. Residents call to report an outage or a leak. The AI takes the address, asks whether the water is off completely or just low, and either flags it for a crew right away or logs it for the morning queue.

B, break it down. The pause a caller feels equals the time to notice they've stopped talking, plus the triage decision, plus, for a caller asking whether a crew is already on the way, a live look-up against the dispatch map, plus the reply starting to speak.
O, own the numbers. 280 milliseconds to notice the caller's done, from the utility's own voice platform. 400 milliseconds for triage. 700 milliseconds for a normal dispatch-map lookup, sourced from the dispatch system's own average. 300 milliseconds for the reply to start.
U, use a range. About 1 second typical for a plain report, p95 target 1.5 seconds. About 1.7 seconds typical for an "is anyone coming" lookup, p95 target 2.5 seconds, the same two-number shape Ondo Mobile landed on.
N, nail the sanity check. The same conversational-pause rule applies, about two tenths of a second is normal, past a second reads as dead air. Delphine Achterberg, who leads product at Rivermark, checked the target against that before signing off on it, not against the utility's old average hold time.
D, direction. Here it isn't the share of calls needing the lookup that swings the number most, it's how many of those lookups land at once. One dispatch-map query takes 700 milliseconds alone. With forty callers hitting it inside the same two minutes during a freeze event, the queue behind that one shared lookup pushes the same query out past three seconds, even though no individual call got more complex. The opposite lesson from Ondo Mobile, where every complex call cost about the same and only the count of them moved the needle.

Swap the trigger and it still runs.
Speed: an interviewer asks how fast this spec needs to exist before freeze-warning season starts. Same equation, solved with wider ranges and no real load data yet, tightened once the first cold snap gives real numbers.
Cost: finance caps the voice line's compute spend. Same equation, run backwards, to find how many concurrent dispatch-map lookups the budget can protect before the p95 target has to loosen.
The model got better: a faster dispatch-map index cuts the lookup from 700 milliseconds to 300. Same ranges on everything else, only one term moves, and mix versus concurrency is still the question worth asking before calling the target safe.

Where people run it wrong.
They size the slow term from a quiet-week average and never ask what happens when forty calls hit it inside the same two minutes.
They set one blended number and let a rare, slow call type hide inside thousands of fast ones.
They fix the target once at launch and never recheck it against how the traffic mix, or the load on a shared system, actually moved.

How to use it live. Say the equation before touching a single number, out loud: the pause, plus the decision, plus the lookup, plus the reply starting. That buys a few seconds to remember which term you can actually defend a number for, and it tells the interviewer the guess is coming, not hiding.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0