U2xAI
…

LLM Evals and AI PRDs

Writing a PRD for an AI Feature Interview Questions

What an AI PRD needs that a normal PRD does not: quality bars, failure behaviour, eval plans and whether to name a model. Many interviews ask you to write part of one live, so practise the sections here.

  • 26 questions with model answers
  • 2 foundational · 15 intermediate · 9 advanced

Foundational Writing a PRD for an AI Feature Questions

  1. What sections does an AI PRD need that a standard PRD does not?Concept
  2. Critique this requirement: the assistant should give helpful and accurate responses.Artifact critique

Intermediate Writing a PRD for an AI Feature Questions

  1. Write the problem statement section for an AI meeting-summary feature.Artifact critique
  2. How do you specify expected behaviour when the output is generated text?Concept
  3. Describe how to document the failure modes section of an AI PRD.Artifact critique
  4. What belongs in the scope section about what the model will explicitly not do?Concept
  5. How do you specify the data dependencies of a feature in a PRD?Concept
  6. How do you document cost assumptions inside a PRD?Calculation
  7. Describe the latency requirements section and how you would derive its numbers.Concept
  8. What does the PRD say about what happens when the model is unavailable?Concept
  9. How do you specify the human-in-the-loop checkpoints in a PRD?Concept
  10. Write the user-facing error handling requirements for an AI feature.Artifact critique
  11. Describe how the PRD should handle the eval spec: inline, linked or separate?Concept
  12. What does the PRD need to say about logging and observability?Concept
  13. What assumptions should be listed explicitly, and why does that matter more for AI?Concept
  14. How do you keep an AI PRD from becoming a technical design doc?Concept
  15. Give me the outline of an AI PRD from memory and justify each section.Interview

Advanced Writing a PRD for an AI Feature Questions

  1. Write the quality bar section for a customer-facing classification feature.Artifact critique
  2. Should the PRD name a specific model? Argue both sides.Interview
  3. How do you write requirements that survive a model version change?Concept
  4. How do you document the guardrails and content policy for a generative feature?Concept
  5. Write the rollout and kill-switch section of an AI PRD.Artifact critique
  6. Describe how you would version a PRD as model behaviour changes underneath it.Concept
  7. What does the success criteria section look like when the metric is subjective quality?Concept
  8. How would you write a PRD for an agent that takes actions on a user's behalf?Case
  9. Which section of an AI PRD is most often missing, and what damage does that cause?Interview