Writing a PRD / Memo / Eval Framework Under Time Pressure

The take-home round isn't testing whether you can write. It's testing whether you've got a structure ready, so that under a time limit you produce something crisp instead of a wall of text. This video hands you two skeletons you can pour any prompt into, the PRD or memo, and the eval framework, plus a method for spending your limited time so the decision lands on page one, not buried on page three.

Here's the thing to hold onto for the whole video. Interviewers reading a take-home skim first and read second. They want the decision or the recommendation up top, then the reasoning that supports it. A memo that makes them hunt for your point fails before they've finished page one. So every deliverable leads with the answer. Everything else in this lesson serves that one rule.

Whether it's called a PRD, a memo, or a two-pager, the bones are the same. In order. First, the decision line, right at the top. One or two sentences: what you recommend, and the single most important reason. The reader should know your position before the end of the first paragraph. This is the highest-impact part of the entire document, so write it first and make it sharp.

Second, problem. What's broken or missing, for whom, and why it matters now. Concrete, with a number if you have one. Not just saying users want more AI, but stating support ticket volume is growing 20 percent a quarter and we can't hire fast enough. Third, users. Who specifically. One primary segment, and their job-to-be-done. Fourth, goal and success metric.

What outcome you're driving, and the one metric that proves it, with a guardrail. This is the part weak PRDs skip. Name the North Star and its guardrail, straight out of the metrics lesson. Fifth, solution. What you'd build. For an AI product, this is where the CIRCLES overlay goes, the data, the model or ML problem, and how it works.

Keep it concrete, but don't drown in implementation. Sixth, eval plan. How you'll know it's good, before and after launch. Offline golden set, online metric, human in the loop for the hard cases. An AI PRD with no eval plan has a visible hole in it. Seventh, risks and mitigations. The failure modes, the guardrails, what could go wrong and what you'd do, including the safety and misuse cases.

This is where judgement shows. And optionally, eighth, rollout. Staged launch, what you watch, the kill switch. You can write this for almost any prompt. The structure carries you even when you're unsure of the content, because each heading tells you what to think about next. Whiteboarding an eval framework for a generative feature, or walking through an eval pipeline, are direct questions that get their own skeleton.

In order. First, define the tasks. What is the feature actually supposed to do? Break it into the concrete tasks you'll evaluate. For a summariser: faithfulness to the source, coverage of the key points, conciseness, format. You can't measure good until you've decomposed it into tasks. Second, build the golden set. A representative set of inputs with known-good outputs or scoring rules.

Cover the common cases and the edge cases, the tricky, the adversarial, the ones you're most worried about. Source it from real usage where you can. And size it so it is runnable often. A few hundred cases is a common starting point. Third, pick the metrics. How each task gets scored. Some are automatic: exact match, does it cite a real source, does it follow the JSON schema.

Some need a model as a judge to rate faithfulness one to five. And some need a human to answer if it is actually helpful. Name which metric goes with which task. Fourth, human in the loop. Auto-eval is cheap and scalable but imperfect, so route the cases where the auto-eval is uncertain, or where the model judge and a human judge disagree, to a human reviewer.

Their labels also improve the golden set over time. That loop is the part that separates a real eval framework from a demo: sample, auto-eval, human review the disagreements, feed it back into the set and the model. Fifth, close the loop. How the results actually drive a decision. A regression on the golden set blocks a release. A pattern of failures becomes new training or prompt work.

The online metric validates that the offline gains were real. An eval that doesn't gate anything is just a dashboard. Say you've got 60 minutes for a written take-home. The mistake is to start writing immediately and run out of time before you reach the risks section. So budget it. First 10 minutes: outline, don't write. Drop the skeleton headings on the page and put one line under each.

Now you've got a complete, if thin, answer. If you ran out of time right this second, you'd still have all the parts present. Next 10 minutes: the decision and the metric. Nail the one-line recommendation and the success metric first, because those are the highest-scoring parts and the easiest to accidentally leave until it's too late. Middle 30 minutes: flesh out solution, eval, and risks.

Add the specifics, the AI overlay, the numbers. Last 10 minutes: cut and sharpen. Move the decision higher if it drifted down. Delete the hedging. Make sure every section is skimmable. A tight two pages beats a rambling five. The principle underneath all of it: outline to completion first, then deepen, so you always have a whole answer and you never leave the highest-value sections unwritten.

Many take-homes end in a panel presentation, so present it the way you wrote it. Decision first, then walk the reasoning, then invite the hard questions. Don't read the document, talk to it. And know your risks section cold, because that's exactly where the panel will push. Calmly owning a real risk, like saying the eval set doesn't cover multilingual yet and here is how I'd extend it, scores much better than pretending there aren't any.

One more thing on the panel, because it trips people up. When someone pushes back hard, they're usually not trying to prove you wrong, they're testing whether you can defend a decision without either collapsing or getting defensive. So the move is to acknowledge the point, say what would change your mind, and then hold your call if the evidence still supports it.

Something like: that's fair. If reply rates on drafted emails came in below hand-written ones, I'd pull the feature back to suggestions only. But the pilot data I'm assuming here points the other way, so I'd ship and watch that metric. That answer shows judgement under pressure, which is the whole thing the panel round exists to measure. Let me pour a prompt into the skeleton.

Write a one-page PRD for an AI feature that helps sales reps draft follow-up emails. Outline first, then fill. The decision line: build an AI draft assist that pre-writes a follow-up email from the call notes and CRM context, editable before send. Recommended because reps spend an estimated five-plus hours a week on follow-ups and skip a lot of them, and draft assist keeps a human in the loop, so we dodge the trust and tone risks of full automation.

Problem: reps drop follow-ups, every missed one is lost pipeline, and writing from scratch is the friction. Users: field sales reps who run eight-plus calls a day. Goal and metric: North Star is follow-up emails sent within 24 hours of a call, per rep. Guardrail is that the reply rate on AI-drafted emails is not lower than hand-written ones, so we're not shipping spammy drafts that reps send and prospects ignore.

Solution: pull call notes plus CRM fields, retrieve the account's recent history, that's RAG because it's per-account and it changes, and generate an editable draft at low temperature for a consistent, professional tone. It's retrieval plus generation, not a fine-tune, because the content is per-account and current. Eval: offline golden set of 200 real calls with a human-written good follow-up, scored for relevance and tone by a model judge plus human spot-checks, and online, the send rate and reply rate metrics above.

Risks: wrong facts about the account, mitigated by grounding in CRM, citing the source field, and keeping it draft only so a human always reviews before send. Tone misfires, mitigated by low temperature and full editability. And never auto-send. That's one page. Decision on top, a metric with a guardrail, the AI overlay woven in, an eval plan, and honest risks with mitigations.

It would survive a panel. Four myths to kill. Bury the recommendation so it feels earned? No. The reader skims. Decision on top always, or it doesn't get read. More pages show more effort? No. A tight two pages beats five rambling ones. They're scoring clarity, not word count. An eval framework is just a metrics list? No. It's tasks, a golden set, metrics, a human loop, and a decision it gates.

The loop and the gate are the parts that matter. And start writing immediately to use all the time? No. Outline to completion first. Writing prose before you have the skeleton is how you run out of time with no risks section. Two skeletons and a clock. The PRD: decision, problem, users, goal plus metric, solution, eval, risks. The eval framework: tasks, golden set, metrics, human in the loop, close the loop.

And the method: outline to completion first, put the decision on page one, then deepen. Memorise this. Two skeletons, one time-box. Outline to completion first, put the decision on page one, then deepen. Structure is what lets you write something crisp when the clock is running.