U2xAI
…

AI Product Metrics and LLM Cost

LLM Latency and UX Tradeoffs Interview Questions

Why latency targets are set at the 95th percentile, when a fast weaker model beats a slow strong one, and why voice interfaces need stricter limits. A common system design topic for AI engineers and PMs.

  • 25 questions with model answers
  • 2 foundational · 11 intermediate · 12 advanced

Foundational LLM Latency and UX Tradeoffs Questions

  1. Describe how streaming changes perceived latency without changing actual latency.Concept
  2. Critique a design that shows a spinner for an unbounded amount of time.Artifact critique

Intermediate LLM Latency and UX Tradeoffs Questions

  1. What is a latency budget and how would you allocate one across a RAG pipeline?Concept
  2. Why do you set latency targets at the 95th percentile rather than the mean?Calculation
  3. How would you decide between a fast weak model and a slow strong one for autocomplete?Case
  4. Explain the UX options available when a response will take 30 seconds.Case
  5. What is time-to-first-token and why is it the metric users actually feel?Concept
  6. Explain when to move work to the background and notify the user later.Case
  7. How would you set a timeout, and what should happen when it fires?Case
  8. Explain the relationship between latency and cost in model selection.Concept
  9. How does latency tolerance differ between a consumer and an enterprise workflow?Concept
  10. What UX patterns make waiting feel productive rather than broken?Concept
  11. How would you communicate a latency regression to users?Case

Advanced LLM Latency and UX Tradeoffs Questions

  1. At what point does latency stop mattering and quality take over?Concept
  2. Describe the latency requirements for a voice interface and why they are stricter.Concept
  3. How do reasoning models complicate a latency budget?Concept
  4. Design the loading experience for an agent that runs for two minutes.Case
  5. How do you handle latency variance rather than average latency?Concept
  6. What is the latency cost of adding a reranking step, and when is it worth it?Calculation
  7. Describe how parallelizing tool calls affects an agent's latency profile.Concept
  8. Describe how you would measure whether latency is costing you conversions.Calculation
  9. How do you budget latency when a third-party API's response time is outside your control?Case
  10. Explain the tradeoff between showing a fast partial answer and a slow complete one.Case
  11. What is the right latency target for an AI search feature, and how did you derive it?Calculation
  12. Set a latency budget live for a feature I describe.Interview

Also in These Skill Collections