CaseIntermediateAI Opportunity & Model Strategy / When NOT to use AI / #16
Give a case where latency requirements alone disqualify a model-based approach.
FLIPS96 percent right and five seconds late reads as zero percent useful
Auracast builds LiveText, an AI captioning and translation feature for live events. Ingvild Wickstrom is the technical director who runs captioning for Northfield Conferences, a three day event with keynotes, panels, and live Q&A. Farrukh Nkemdirim is the Auracast PM who has to explain, after Day 2, exactly why an accurate model still failed live.
The direct answer
A live panel discussion with fast, overlapping speakers, where a model's honest processing latency runs past 5 seconds behind real speech, disqualifies that model outright, no matter how accurate its transcript eventually is. Live captions that arrive after the room has already moved on aren't slightly degraded, they're unusable. Set the hard latency budget first, then only compare models that can plausibly hit it, even at a real accuracy cost.
Do this, in order
Set the real-time latency budget before comparing any model on accuracy.Why: a caption nobody can read in time isn't a slightly worse caption, it's a failed one, regardless of word accuracy.
Test latency under real, overlapping, fast speech, not calm single-speaker audio.Why: an offline benchmark's controlled conditions hide exactly the failure mode that shows up live.
Split scope by segment type once real numbers show where the budget breaks.Why: the same event has genuinely different latency risk in a scripted keynote versus a live panel.
Keep a human fallback for segments the model can't reliably hit the budget on.Why: this isn't a permanent ban on AI captioning, it's matching the tool to where it can actually keep up.
State the accuracy trade-off out loud when the budget forces a simpler model.Why: a faster, slightly less accurate model that stays inside the budget beats an accurate one that arrives too late to read.
Recheck the split whenever the model or the event format changes.Why: latency budgets and real speech patterns both move. A split made once isn't a split made forever.
How to answer this, stage by stage
Nobody is scoring whether you know what "latency" means. They're scoring whether you'd have caught this before a deaf attendee filed a formal complaint.
Stage 1
Scope it to one real case, not a lecture on real-time systems
Say it like this
"Let me give you a real case. A live captioning model, 96 percent accurate on a calm benchmark, running at a conference. On a fast, overlapping panel discussion, its latency spiked to 5.8 seconds behind live speech. That's the case where latency alone kills it, regardless of accuracy."
Why this works
Grounds the answer in something checkable instead of a general claim about AI and speed.
Stage 2
Say your structure out loud before any content
Say it like this
"I'll run this as FLIPS. Find the person it happened to. Locate the habit they had before. Identify the flip, the two settings with no middle. Pinpoint the old decision behind it. Show the replay with a real number on the clock."
Why this works
Signals a repeatable diagnostic method instead of a single anecdote pulled from memory.
Stage 3
Reframe the question: this isn't about accuracy at all
Say it like this
"The real question isn't 'is this model good enough.' It's 'can this model possibly respond inside the time a live audience actually has.' Those are different questions, and only one of them showed up in the offline benchmark."
Why this works
This is where a strong answer separates from a list of accuracy metrics.
Stage 4
Give the one decision: set the latency budget first, as the gate
Say it like this
"Here's what I'd actually do. Before comparing any model's accuracy, set the hard real-time budget, about 3 seconds for live captions to stay readable. Only then compare models that can plausibly hit it under real, overlapping speech. Accuracy is the second question, not the first."
Why this works
This is the direct answer to the question, stated as something you'd actually do, not just describe.
Stage 5
Prove it with the compressed failure
Say it like this
"This is exactly what happened at a real conference. The model looked great on a calm benchmark. On a heated three way panel, overlapping speech pushed its latency to 5.8 seconds, and the captions read a whole exchange out of order. A deaf attendee filed a complaint. That's not a bug to patch, it's the model doing the one thing latency requirements were always going to catch."
Why this works
This is where the FLIPS story lives, compressed to the moment that actually proves the decision.
Stage 6
Say what you'd measure, and the trade-off it forces
Say it like this
"I'd measure the percent of caption spans that blow past the 3 second budget, split by segment type, not just an average across the whole event. And I'd say plainly: a faster, slightly less accurate model that stays under budget beats an accurate one that arrives too late to read. That's the trade being made, out loud."
Why this works
Shows you'd catch this with real measurement, not gut feel, and names the AI-specific trade-off honestly.
Stage 7
Say what you'd leave alone, then close on one line
Say it like this
"Scripted keynotes stay on the AI captioning, the pacing there is slow and single speaker enough that the model comfortably clears the budget. It's only fast, overlapping, live speech where the budget disqualifies it outright. Set the budget first, and let it decide, segment by segment."
Why this works
Closes with real judgment instead of writing off AI captioning everywhere, and restates the direct answer in one breath.
Let's learn
LiveText is a tool that listens to a live event and generates captions and translations in real time, so attendees who are deaf, hard of hearing, or listening in a second language can follow along as people speak.
The method, in five rows. The I step is the one this whole case turns on.
Before Day 2, Ingvild ran LiveText across every session of the conference, all three days, on the strength of Auracast's own number: 96 percent word accuracy, 2.1 seconds of average latency, measured on a calm, single-speaker benchmark. It held up fine through Day 1's keynotes, slow, scripted, one voice at a time.
Percent of live captions exceeding the 3 second readable budget, by session type
Keynote, scripted, one speakerPanel, live, overlapping speech
Same model, same event, radically different real-world latency depending on how the speech itself behaves. The average across the whole event hid this completely.
Then came Day 2's afternoon panel: three founders, genuinely disagreeing, talking over each other. The model had to process and untangle overlapping audio before it could emit a stable caption line, and once behind, it never caught back up before the panel ended.
96 percent accurate and five and a half seconds late isn't a slightly worse caption. To the person reading it live, it's a transcript of a conversation that already ended.
Here's the turn: the problem was never the model's accuracy. The captions, once they finally arrived, were mostly correct. The turn is that correct and on time are two completely different bars, and a live audience only ever gets to benefit from the second one.
Knowledge spark: why does overlapping speech blow up latency specifically?
A speech model has to figure out who's talking and untangle the audio before it can commit to a caption line. With one calm speaker, that's fast. With three people talking over each other, the model often has to wait for a pause to resolve the overlap, and every extra second of waiting adds directly to what the audience sees on screen.
At its worst, this cost showed up as captions that scrambled the order of a real exchange: a rebuttal appeared on screen before the claim it was answering, because the rebuttal's audio had been easier to resolve. A deaf attendee, relying entirely on the captions, filed a formal complaint that the panel had made no sense.
The choice I would take back
Running LiveText across the entire three day event on one blended accuracy number, without ever testing its latency under real, overlapping, fast-paced speech. That made sense when the only number anyone had was the calm benchmark. It stopped making sense the moment a live panel proved that number didn't describe every session equally.
What I would leave alone: the keynote sessions stayed exactly as they were, AI captioning on, no fallback needed. Slow, scripted, single-speaker audio comfortably clears the 3 second budget, so there was never a real problem there to fix.
The lesson: a model's accuracy number and its latency number answer two different questions, and only the second one determines whether a live audience gets to use it at all. Test latency where the real speech is hardest, not where it's calmest.
Now here is the same thing as a story
The short version above is what you'd say out loud in the room. Read this one for what it actually felt like to watch a caption fall five seconds behind and never catch up.
Ingvild Wickstrom had run live captioning for conferences for six years, first with human stenographers, more recently with AI tools layered in. She knew, by feel, when a captioning setup was working: the text on screen matched what she'd just heard, close enough that nobody in the room noticed a gap.
Day 1 went well. Three keynotes, each one speaker, measured pacing. LiveText held its 2.1 second lag the whole day, close enough that Ingvild stopped watching the caption screen entirely by the second talk.
The benchmark's calm number and the panel's real number were never going to be the same measurement.
Day 2, 2:40pm, the "Founders Under Fire" panel. Three people who genuinely disagreed, talking over each other within the first two minutes. Ingvild had the caption screen open on a side monitor, mostly out of habit.
The gap started small. A second behind, then two. By the fourth minute it was past four seconds, and the captions were now describing an exchange that had already resolved on stage. By the time one panelist's sharp rebuttal appeared on screen, the room had already moved past it, laughing at something else entirely.
We didn't lose four points of accuracy. We lost the thing that made the captions captions at all: arriving while the conversation they described was still happening.
The complaint came in that evening, from an attendee who'd relied entirely on the captions. She hadn't followed a heated, funny, substantive panel. She'd followed a transcript that felt like it was describing a different, more confusing conversation, because half the ordering was scrambled by which speaker's audio happened to untangle faster.
Nothing about Day 1 predicted Day 2. That's what made the trigger land as a genuine surprise instead of a slow decline anyone was watching for.
Ingvild never had a fixed number in her head for when a captioning setup had failed. She had a feeling with two settings: it feels like it's keeping up, or it very obviously isn't. Day 1's 2.1 seconds never touched that second setting. Day 2's 5.8 did, in front of a room, with a name on the complaint.
Back when LiveText was first scheduled for the whole event, the decision wasn't unreasonable. The only real number anyone had was the calm benchmark, and it looked strong. It stopped being reasonable the moment a genuinely fast, overlapping conversation showed that number had never described panels at all.
One blended trust setting for the whole event was the actual mistake, not the model's accuracy.
Here's the replay: same model, same panel format, but with the segment split already in place. LiveText stays on for the keynotes. For "Founders Under Fire," a human stenographer runs instead, live, at the pace the room actually needs, and LiveText quietly transcribes in the background as a backup record, off the screen where its lag can't confuse anyone in real time.
The fork that should have existed from day one, drawn out plainly.
One version of Day 2 ends with a complaint and a scrambled transcript nobody could follow live. The other ends with captions that stayed a beat behind real speech, the way a skilled human stenographer's always have, close enough that nobody in the room ever had to notice a gap.
What I'd tell myself, watching that gap open on the side monitor: a benchmark number measured in the calmest possible conditions was never going to tell me what happens the one time speech actually gets hard.
The five steps, if you want to remember itNot a script for defending a model that's already too slow. FLIPS is what catches the gap between a calm benchmark and a real, overlapping room before an attendee has to file a complaint about it.
F
Find the person. Who does this every day, and what are they good at?
Ingvild Wickstrom, six years running live captioning, first with human stenographers, now layering in AI tools. She knows the feel of captions keeping up.
A real person with real competence, not an abstract "the system."
L
Locate the habit. What did they trust, and why did it make sense?
Scheduling LiveText across the whole three day event, on the strength of one blended accuracy and latency number measured on calm, single-speaker audio.
The habit made real sense with the only number anyone had at the time.
I
Identify the flip. The two settings, with no middle.
Trusting LiveText everywhere, on one blended number, versus splitting scope by segment type once real, overlapping-speech latency proved the blended number was hiding a genuine failure case.
This is the direct answer to the question, as an actual behavior change, not a mood.
P
Pinpoint the old decision. What only worked because latency hadn't been tested yet?
Choosing to schedule LiveText across every session based on a calm benchmark, without ever testing its latency under real, fast, overlapping speech before the live event.
The gap was always there. Nobody had looked for it in the one place it would actually show up.
S
Show the replay. Same day, different design, with a number on the clock.
Keynotes stay on LiveText, comfortably inside the 3 second budget. "Founders Under Fire" runs a human stenographer live, with LiveText quietly recording in the background instead of driving the on-screen captions.
Same event, same model, and the one segment that couldn't hit the budget gets a different tool instead of a lower bar.
The recap, one line per letter: find is a technical director with a real feel for what "keeping up" means, locate is the habit of trusting one blended benchmark number across every session, identify is the flip from trusting it everywhere to splitting scope by real latency, pinpoint is the decision to never test overlapping speech before the live event, and show is the replay where the model keeps the segments it can actually hit the budget on.
And if you want to be sure it really works, try it somewhere elseSame method, a customer support voice line instead of a conference stage. This time the flip family is substitution, not scope.
Baraka Osei runs the support desk at Ferrowave, a logistics company. Their AI voice assistant answers routine shipment-status calls in under a second, comfortably inside the 2 second budget callers expect from an automated line. When call volume spikes during a weather delay, though, the queue backs up and the same model's real response time climbs past 6 seconds per turn as it competes for shared compute. Baraka's team used to route every call type to the assistant first. Now, during measured high-volume windows, they route the highest-friction call types, active complaints, straight to a human, and save the assistant for the routine lookups it can still answer inside budget. Mapped onto FLIPS: find is Baraka, who watches the queue dashboard daily. Locate is routing every call to the assistant first, rain or shine. Identify is the substitution flip, not scope this time: under load, the assistant gets reserved for the easy cases it can still answer fast, while the hard, high-friction calls get saved for a human, the opposite of what you'd want if speed were the only concern. Pinpoint is never having tested the assistant's response time under real peak load before a weather event proved the gap. Show is the replay where routing by real-time load, not a fixed default, keeps every call inside a budget that actually holds.
LiveText latency across "Founders Under Fire," minute by minute
Under budgetOver budget
It crossed the 3 second budget by minute 2 and never came back under it for the rest of the panel. That's a disqualifying case, not a slow one.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Skip straight to "set the latency budget before comparing accuracy, test it under real overlapping speech," and stop.
Cost: no budget for a human stenographer fallback. Keep the AI captioning everywhere, but make the lag visible on screen so the audience knows to expect a delay instead of trusting a false real-time feel.
The model got better, for real: say a future version resolves overlapping speech in under a second. Retest the real number under real conditions before trusting it, a claim like that needs its own evidence, not just a vendor's roadmap.
Where people run it wrong.
They test latency only under calm, controlled conditions, and never under the fast, overlapping speech that actually shows up live.
They average latency across an entire event, hiding a segment where the budget fails completely inside a number that looks fine overall.
They treat a latency failure as an accuracy problem to fix with more training data, when the real issue is architectural, not a data gap.
How to use it live. The moment an interviewer describes a real-time AI feature, ask yourself first: what's the actual time budget the user has, and has it been tested under the hardest real conditions, not the calmest ones. That question buys real thinking time, and it's usually exactly where the disqualifying case is hiding.
Flashcards (tap any card to flip it)
1 · THE FLIP FAMILY
What flip family is this?
Tap to flip
ANSWER
The scope flip: runs it on everything, then runs it on a small slice. Ingvild moves from AI captioning across the whole event to AI captioning only on segments that can actually stay inside the latency budget.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Ingvild Wickstrom, the technical director running live captioning for Northfield Conferences, using Auracast's LiveText tool.
3 · THE HABIT
What did they stop doing because it worked, on Day 1?
Tap to flip
ANSWER
She stopped watching the caption screen closely during the keynotes, once the AI captions comfortably held their 2.1 second lag through the whole first talk.
4 · THE FLIP, IN THIS STORY
What's the two-setting switch here?
Tap to flip
ANSWER
Trusting LiveText across the entire event on one blended number, versus splitting scope by segment type once real, overlapping-speech latency proved the blended number was hiding a genuine failure.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Scheduling LiveText across every session based only on a calm, single-speaker benchmark, without ever testing its real latency under fast, overlapping speech before the live event.
6 · THE NUMBER
Fill in the blank: LiveText's latency on the panel peaked at ___ seconds, well past the ___ second readable budget.
Tap to flip
ANSWER
5.8 seconds, past a 3 second budget. Once it crossed that line by minute 2, it never recovered for the rest of the panel.
7 · THE REPLAY
Same bad day, new design, what changes?
Tap to flip
ANSWER
Keynotes stay on LiveText. "Founders Under Fire" runs a live human stenographer instead, with LiveText recording quietly in the background, off the screen where its lag can't confuse anyone in real time.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this same question again for a different product, with a different flip family. Which product, and which family?
Tap to flip
ANSWER
Ferrowave's support voice line, using the substitution flip: under peak load, the assistant gets reserved for easy calls it can still answer fast, while hard calls go to a human instead.
Check yourself Score: 0 / 0
Multiple choice
1. What was the flip in Ingvild's story, and what were its two settings?
A. Checking captions occasionally, then checking every single one.
B. Running LiveText across the whole event, then splitting scope so it only runs where it can hit the latency budget.
C. Using a human stenographer for everything, then switching fully to AI.
D. Ignoring the complaint, then apologizing to the attendee.
Show hint
Look at the I step in "The five steps, if you want to remember it."
Show answer
B. A scope flip: the unit of trust shrinks from the whole event to only the segments that can actually stay inside the real-time budget.
True or false
2. True or false: Ingvild could have fixed this by asking LiveText's team to just check the captions more carefully before they went live.
True
False
Show hint
Live captions have to appear in real time. There's no "check it first" step possible.
Show answer
False. Live captioning has no review step before the audience sees it. The fix has to be architectural, matching the tool to what it can hit in real time, not a verification habit.
Fill in the blank
3. Fill in the blank: across the full event, ___ percent of caption spans in panel and Q&A sessions exceeded the 3 second budget, versus only ___ percent in keynote sessions.
Show hint
Look at the bar chart in "Let's learn."
Show answer
61 percent, versus 4 percent. The same model, the same event, but a radically different real-world failure rate depending on how the speech itself behaved.
Short answer, where it wouldn't matter
4. Name a place in this same event where the latency issue did NOT cause a flip, and say why not.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: The keynote sessions. Slow, scripted, single-speaker audio comfortably cleared the 3 second budget, so there was never a real gap to fix there.
Short answer, apply it yourself
5. Pick a real-time product you use yourself. What's its actual time budget, and has anyone ever tested it under the hardest real conditions instead of the calmest ones?
Show hint
Think of anything that has to respond while you're waiting, live: a voice assistant, a live sports score overlay, a video call's noise cancellation.
Show answer
Model answer: A video call's live noise cancellation has a budget of a few tens of milliseconds to stay unnoticeable. It's usually tested on clean audio, not the hardest real case: three people talking over a barking dog and a bad connection at once.
Short answer, work the number
6. If the panel's latency had peaked at 3.4 seconds instead of 5.8, just over the budget instead of nearly double it, would the direct answer still hold that the model should be disqualified for that segment?
Show hint
Think about what actually makes captions unusable: crossing the budget at all, or crossing it by a specific margin.
Show answer
Model answer: Yes, mostly. Once captions consistently fall behind the budget, even by less, the audience experience degrades the same way: reading a transcript of a conversation that's already moved on. The margin changes how bad it looks, not whether the segment should get a fallback.
Before you close the answer
Why this works
Tests whether you'll separate a model's accuracy from its real-time viability, and whether you know to test latency under the hardest real conditions, not the calmest ones a vendor benchmark usually reports.
Follow-up traps
"Couldn't they just add more compute to fix the latency?" Response: some, but overlapping-speech disambiguation is partly an architectural bottleneck, not just a compute one, so more compute helps less than it would for a purely compute-bound task.
"Isn't 3 seconds an arbitrary budget?" Response: it's grounded in real accessibility guidance for live captioning readability, and it's exactly the number Day 2's real complaint confirmed in practice, not a number picked to make the story work.
If pressed
The model's overlapping-speech slowdown came from needing a short pause in the audio to confidently attribute each phrase to a speaker before committing a caption line, a real architectural constraint, not a data or training gap that more examples would fix.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
