AI UX Design Interview Questions
Designing products whose output is sometimes wrong: showing uncertainty, failing gracefully, human review, feedback loops, explainability and onboarding. Asked of product designers and UX researchers on AI teams, and of AI product managers. 132 questions across 6 subtopics, each with a model answer.
UX for AI Uncertainty and Confidence
How to show users that the AI might be wrong without making the product annoying: confidence display, encouraging verification, and what changes when the model is unsure. Relevant to AI PMs and product designers.
- When should you show a confidence score to a user, and when should you hide it?
- Describe three ways to communicate uncertainty without displaying a number.
- What is the risk of showing a percentage confidence that users cannot interpret?
- Design the UI for a feature that is 70 percent confident in its answer.
- Explain how hedging language in generated text affects user trust.
- How would you design an interface that encourages verification without being annoying?
- What visual patterns signal that content was AI-generated?
- Critique a design that presents an AI answer with the same authority as a database lookup.
Designing for AI Failure
What the product should do when the model returns nothing useful, how to fall back to a non-AI version instead of breaking, and how to help a user recover after an agent takes a wrong action.
- What should happen in the UI when the model returns nothing usable?
- Design the fallback experience for an AI feature when the provider is down.
- Explain the difference between failing loudly and failing silently, and which you prefer.
- How do you design a feature that degrades to a non-AI version rather than breaking?
- Describe three failure modes to design for before launch.
- What error message would you write for a model timeout, and what would you avoid saying?
- How should the product behave when the model produces output that fails a safety filter?
- Design the recovery path for a user whose agent took a wrong action.
Human in the Loop Design
When a person should review the AI's work, how to decide which cases go to a human, why review fatigue breaks the design, and how to build approval flows for agents.
- When should a human be required to approve an AI action rather than merely able to?
- Design the review interface for a human checking 200 AI-generated outputs an hour.
- Explain how review fatigue undermines a human-in-the-loop design.
- What is the difference between human-in-the-loop and human-on-the-loop?
- How do you decide which cases get routed to a human?
- Describe a confidence-based routing policy and its failure mode.
- How do you measure whether the human in the loop is adding value?
- Design an approval flow for an agent that sends external emails.
AI Feedback Loops and Data Flywheels
How to collect feedback that actually improves the product: why thumbs up and down are weak, how to avoid capturing only complaints, and how feedback becomes better data over time.
- Design the feedback mechanism for an AI feature where users rarely click thumbs down.
- Explain the difference between explicit and implicit feedback signals.
- What implicit signals tell you an output was bad?
- How do you avoid a feedback loop that only captures complaints?
- Describe how you would turn user edits into a quality signal.
- What is the latency between collecting feedback and improving the product, and how do you shorten it?
- Critique a thumbs up and down widget as a feedback mechanism.
- How do you design feedback that is useful for debugging rather than just scoring?
AI Trust, Transparency and Explainability
What users need to see before they trust an AI recommendation, how to label AI output honestly, and how to design opt outs that respect users without breaking the product.
- What does a user need to see to trust an AI recommendation?
- Explain the difference between explainability and transparency in a product context.
- How do citations change user behaviour, and what happens when they are wrong?
- Design the disclosure that tells a user they are talking to an AI.
- When does showing the model's reasoning help, and when does it reduce trust?
- Critique a design that surfaces a chain of thought to end users.
- How much should you tell users about which model powers a feature?
- Describe how you would design for a user who wants to audit an AI decision.
Onboarding Users to AI Features
How to introduce someone to an AI feature for the first time, how much friction is right, and how to re-onboard existing users when the AI's behaviour changes.
- Design the first-run experience for a user who has never used an AI feature.
- How do you teach users what the AI can and cannot do without a manual?
- Explain the role of example prompts in onboarding and their downside.
- What is the risk of an onboarding that oversells capability?
- How would you set expectations about errors during onboarding?
- Describe progressive disclosure for a complex AI feature.
- Critique an onboarding that starts with a blank chat box.
- How do you onboard a user into an agent product where the AI acts on their behalf?