ConceptAdvancedResponsible AI & Advanced Practice / Building an AI PM portfolio / #12

How would you document the tradeoffs you rejected, and why does that matter?

PICK the portfolio piece is Civora, an AI tool that matches small nonprofits to grants they qualify for

Harborlight Civic Fund is a fictional civic-tech nonprofit that helps small community groups find and apply for grants. Solveig Tjaden built Civora there as her portfolio project: it reads a group's project description and matches it against open grant programs, then drafts the eligibility case. Tobias Wexler, a former AI PM, mentors her through the write-up.

The direct answer
Log every rejected tradeoff as a short decision record, not a story: the option you picked, the option you rejected, and the exact number or signal that would make you switch back. A record like that is checkable. A paragraph explaining your reasoning after the fact is not, and a sharp interviewer can tell the difference in one follow-up question.
Do this, in order
  1. Write each rejected tradeoff as a decision record with a stated kill signal, not a narrative explanation.Why: a record is checkable. A story about your reasoning is not.
  2. Log the record while you're building, not after you ship.Why: real-time reasoning decays fast, and a reader can tell reconstructed reasoning from the real thing.
  3. Attach the actual evidence behind the reject, an eval score, a screenshot, not just a description of it.Why: a description is a claim. An artifact is proof.
  4. Keep each record short: one option, one reason, one signal.Why: length doesn't add credibility. Specificity does.
  5. Skip the formal record for ideas dropped in the first exploratory hours, before anything shipped.Why: not every abandoned idea deserves the same weight, only the ones a reader would reasonably ask about.
  6. Revisit old records when new evidence actually flips one.Why: a kill signal nobody ever checks again isn't a signal. It's decoration.

How to answer this, stage by stage

Nobody is grading whether you can list what you didn't build. They're grading whether your list is something they could actually check.

Stage 1
Scope it to one real project
Say it like this
"I'll answer this for Civora, a tool that matches nonprofits to grants they actually qualify for."
Why this works
Grounds an abstract documentation question in one real portfolio page.
Stage 2
Say your structure out loud
Say it like this
"I'll use PICK. Position, my pick on how to document it. Impact, who it costs. Cost asymmetry, which cost is hidden. Kill criteria, what would change my pick."
Why this works
Signals this is itself a tradeoff decision, not just a formatting preference.
Stage 3
State your position before the reasoning
Say it like this
"My pick: a short decision record with a stated kill signal, over a narrative paragraph explaining my reasoning."
Why this works
Commits to an answer before hedging into "it depends," which is what the question is actually testing.
Stage 4
Name who each error costs
Say it like this
"A vague story costs the reader a few extra minutes of suspicious follow-up questions. A structured record costs the writer a few extra minutes upfront, while it's still fresh."
Why this works
Names both sides in real minutes, not vague discomfort.
Stage 5
Say which cost is hidden
Say it like this
"The narrative's cost hides until the toughest interview question, and by then it's too late to fix. The record's cost is visible and paid once, while you're writing it."
Why this works
This is the heart of PICK: naming which error is cheap and visible versus hidden and expensive.
Stage 6
Give the kill criteria, and close
Say it like this
"I'd switch back to a narrative for anything abandoned in the first hour of exploring, before I'd built anything worth defending. Past that point, it's a record, every time."
Why this works
Shows this isn't a stubborn rule. It's a real pick with a stated edge.

Let's learn

A rejected-tradeoff record is a short note that says what you didn't build, and exactly what would have to be true for you to reconsider it.

Tobias Wexler mentors AI PM candidates through their portfolio write-ups. For a long while, he took a narrative paragraph like "I considered X, but Y felt safer" at face value, since it read like real reasoning.

Knowledge spark: what's a kill signal? The exact number or event that would make you reverse a decision. Not "if it stops working," but "if false positives cross 5 in 100." Without one, a decision can never actually be tested again.

Lately, across dozens of mentee reviews, Tobias has noticed something. A narrative-style rejection now costs him about eight minutes of probing before he trusts it, if he ever does. A structured record, with a real number in it, costs him about ninety seconds, because the record is basically pre-answered.

Minutes to write versus minutes an interviewer spends probing it
12 min 6 min 0 Write 3 min Probe 8 min Narrative style Write 12 min Probe 1.5 min Structured record
The narrative is faster to write and slower to trust. The record flips that: more time up front, almost none spent defending it later.

More narrative detail never made Tobias trust a rejection more. A longer story explaining a decision he can't check is just a longer thing to stay suspicious of.

The story that took three minutes to write cost eight minutes of an interview defending it.

At its worst: a candidate who genuinely made the right call loses an entire interview's momentum, not because the decision was wrong, but because the way it was written gave the interviewer nothing to check.

The decision that mattered Log the option, the reason, and the kill signal, in that order, the moment you reject something. Not weeks later, when you're writing the portfolio page from memory.

What I would leave alone: the tone and personal voice in the rest of the write-up still matter and should stay free-form. Only the rejected-tradeoff claims themselves need the structured format.

The lesson: a record beats a story, not because it's more honest, but because it's checkable. Checkable is the only kind of honest an interviewer can verify in twenty minutes.

Now here is the same thing as a story

The short version above is what you'd say defending your documentation habit out loud. Read this one for how Solveig actually built hers.

Every Sunday night, Solveig Tjaden used to update a running note of everything Civora got wrong that week, a habit left over from three years as a grants coordinator, where she'd learned which rejections actually stung a small nonprofit and which ones didn't matter at all.

In her first weeks building Civora, she wrote full paragraphs the moment she made a real call. One entry, from an early Tuesday, ran six sentences on why she'd picked precision over recall for the eligibility matcher.

Hand sketched metaphor scene titled A story versus a record. Left, a document icon labeled Story, caption told once. Right, a gauge icon labeled Record, caption checked twice.
A story only has to sound right the first time someone hears it. A record has to survive being checked a second time.

The habit thinned in three beats. Three weeks in, tired on a Sunday night, she started summarizing each tradeoff in one line instead of a paragraph. By the time she wrote her actual portfolio page, months later, she wrote from memory: "we chose precision over recall because false positives hurt trust more." It sounded reasonable. It also wasn't quite the reasoning she'd used in the moment, and she'd genuinely forgotten the number behind it.

Then Tobias, reading her draft, asked one plain question: "What number made you pick precision over recall? What would have to happen for you to flip that?" She didn't have an answer beyond "it felt safer."

Hand sketched comparison diagram titled The asymmetry, drawn. Left, a small question mark box icon labeled One question, caption an awkward moment. Right, a large red box icon labeled Trust drops, caption whole page doubted.
One unanswerable question rarely stays contained to itself. It spreads doubt across every other claim on the page.

Solveig spent the next four hours digging through old Slack messages and a private spreadsheet she'd kept of Civora's weekly eval runs, a workaround she'd built for herself since the tool itself kept no history of its own. She found the real number: a false positive, a wrongly suggested grant, cost a small nonprofit about 40 minutes re-verifying eligibility it never actually had, while a false negative, a real match Civora missed, cost about 10 minutes of extra manual search.

Hand sketched timeline titled When the real tradeoffs happen. Four milestones: kickoff week 1, first eval run week 3, mid-build pivot week 5 highlighted, launch week 8.
The real decision happened in week 5. The write-up describing it got written months after launch, from memory alone.

The real cost was never the four hours spent rebuilding the record. It was that without her old spreadsheet, that number would have been gone for good, replaced by a guess that sounded just as confident.

Back on that tired Sunday night, cutting her notes down to one line each had felt harmless. It saved her maybe fifteen minutes a week, and the project felt basically done. It stopped feeling harmless the day someone asked her to defend a number she no longer had.

Hand sketched decision tree titled Reading a rejection two ways. Root, a logged rejection, branching to written after launch leads to reads like spin, written in the moment leads to reads like judgment.
The exact same sentence reads two different ways, depending only on when it got written down.

The next rejected tradeoff Solveig logged, choosing a smaller local grants dataset over a larger national one to train the matcher, took her twelve minutes to write as a full record, kill signal included. When Tobias read it, his one real follow-up question was already answered in the same paragraph, and he moved to her next project in under two minutes.

The old note asked Tobias to trust her memory. The new one gave him the exact number her memory used to hold.

I cut the full version to save fifteen minutes a week. It cost me four hours later, and it nearly cost me the one number that proved I'd made a real call at all.

PICK, in one screenNot a style preference. PICK is what forces a documentation choice to name its own hidden cost.

P
Position. Your pick, stated first.
A structured decision record with a stated kill signal, over a narrative paragraph.
Committed before the reasoning, not after.
I
Impact. Who feels each choice, in real minutes.
A vague story costs the reader minutes of suspicious follow-up. A record costs the writer minutes upfront.
Names both sides in units a listener can picture.
C
Cost asymmetry. Which cost is hidden.
The narrative's cost hides until the hardest interview question. The record's cost is visible and paid once, upfront.
The heart of the whole pick: optimize against the hidden one.
K
Kill criteria. What would flip the pick.
Skip the formal record for ideas dropped in the first exploratory hours, before anything real had shipped.
Separates a real decision from a rule applied blindly everywhere.
Hand sketched flow diagram titled Building one decision record. Five boxes: note option, state reason, name signal highlighted, attach proof, file it.
Naming the exact signal is the step most candidates skip, since it forces a number they don't yet have.
Hand sketched icon list titled What a decision record needs. Items: the option rejected, the real reason, the kill signal, the evidence, the date it happened.
Five short lines. All five fit on one sticky note, and that's the point.
Reader trust score, by number of past records with a kill signal
10 5 0 Enough, past here 0 records 1 2 3 4+
The jump from zero to two real records does almost all the work. Writing a fifth adds almost nothing, past the point marked here.

The recap, one line per letter: position is a structured record over a story, impact is minutes spent by reader versus writer, cost asymmetry is a hidden cost versus a visible one, and kill criteria is skipping the formal record for the earliest, still-forming ideas.

And if you want to be sure it really works, try it somewhere elseSame four letters, a container port instead of a grants office. The kill signal this time is a missed-anomaly rate, not a minute count.

Kessling Port Authority is a fictional container port. Rutger Voss built an AI tool there that prioritizes which containers get a full inspection. Nadia Furstenberg reviews his portfolio write-up.

Mapped onto PICK: Rutger's position was picking a faster, cheaper anomaly-detection model over a slower, more thorough one. The impact: inspectors feel the visible cost, an occasional slow queue if the faster model misses something subtle, while the port feels the hidden cost, a shipment held for days over a false alarm from an overcautious slow model. The cost asymmetry: false alarms from the slow model cost the port real money and time on every occurrence, while a true miss from the fast model is rare, but far more serious when it happens. Nadia pushed him past a vague "the fast model felt like the right balance" until he named a real kill criterion: if the missed-anomaly rate in production ever crossed one flagged case per thousand containers, he'd switch back to the slower model, no exceptions.

Hand sketched labeled parts diagram titled The container queue write-up. Center document icon labeled Rejected tradeoff, with four callouts: kill signal set, missed rate named, faster model kept, reviewer's question.
A named rate, not a feeling, is what let Nadia stop probing after one read.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "I log the option, the reason, and the number that would flip it, right when I make the call," and stop.
Cost: there's no time to write a full record for every idea you drop in a single afternoon of exploring. Save the format for the choices that actually shipped, or that a reader would reasonably ask about.
The model gets better, for real: if Civora's matching accuracy improves later, the old kill signal doesn't disappear, it just means the number that would flip it should get re-checked, not deleted.

Where people run it wrong.
They write a narrative explanation because it feels more personal, and it reads like spin the moment someone probes it.
They write the record weeks after the decision, from memory, and quietly change the reasoning without noticing.
They state a kill signal once and never check it again, which makes it decoration, not a real decision rule.

How to use it live. When someone asks how to document a rejected tradeoff, ask yourself one question first: what number would make me reverse this. Say that number out loud before you say anything else about your reasoning.

Flashcards (tap any card to flip it)

1 · THE FRAMEWORK
What framework fits "how would you document the tradeoffs you rejected"?
Tap to flip
ANSWER
PICK: position, impact, cost asymmetry, kill criteria. Kill criteria is what turns a documented opinion into a testable decision.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Solveig Tjaden, a former grants coordinator who built Civora, a grant-matching tool, for her portfolio.
3 · THE HABIT
What did Solveig stop doing across her weekly notes?
Tap to flip
ANSWER
Writing full paragraphs on the tradeoffs she made, the moment she made them. She trimmed each one down to a single line to save time.
4 · THE MECHANISM
What's the cost asymmetry between a narrative and a structured record?
Tap to flip
ANSWER
The narrative's cost hides until a tough interview question. The record's cost is visible and paid once, upfront, while it's written.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Trimming full paragraphs to one-line notes to save about fifteen minutes a week, once the project started to feel finished.
6 · THE NUMBER
Fill in the blank: a false positive cost a small nonprofit about ___ minutes re-verifying eligibility it never had.
Tap to flip
ANSWER
40 minutes, versus about 10 minutes of extra search for a missed real match. That gap is why Solveig picked precision over recall.
7 · THE REPLAY
Same kind of tradeoff, logged properly this time. What changes?
Tap to flip
ANSWER
A twelve-minute record with a kill signal answers Tobias's one real follow-up question in the same paragraph he's reading.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which one, and what's the kill signal?
Tap to flip
ANSWER
Rutger Voss's container-inspection tool at Kessling Port Authority. The kill signal: one missed anomaly per thousand containers.

Check yourself Score: 0 / 0

True or false
1. True or false: this answer recommends writing a longer, more detailed paragraph explaining your reasoning behind a rejected tradeoff.
  • True
  • False
Show hint
Look at the grouped-bar chart comparing narrative style to a structured record.
Show answer
False. The fix is a shorter, structured record with a named kill signal, not a longer story. More prose never made Tobias trust a claim more.
Multiple choice
2. Why does a structured decision record end up cheaper overall than a narrative explanation, even though it takes longer to write?
  • A. Because narratives use more expensive words.
  • B. Because reviewers always prefer shorter documents, regardless of content.
  • C. Because the narrative's real cost is hidden until a tough follow-up question, while the record's cost is visible and paid once, upfront.
  • D. Because a record never requires a rejected alternative to actually exist.
Show hint
Look at the C step: cost asymmetry.
Show answer
C. This is the entire point of PICK: optimize against the cost that's hidden, not the one that's merely visible.
Fill in the blank
3. Fill in the blank: reconstructing her lost decision record cost Solveig about ___ hours, digging through old messages and a private spreadsheet.
Show hint
Look in the story, right after Tobias's question.
Show answer
Four hours. Nearly lost for good, since her tool kept no history of its own and the number lived only in her private workaround.
Short answer, where it wouldn't matter
4. Name a kind of rejected idea where this exact record format is genuine overkill.
Show hint
Look at the kill criteria step.
Show answer
Model answer: Something dropped in the first exploratory hours, before anything real had been built. It doesn't yet deserve the same formal weight as a shipped decision.
Short answer, apply it yourself
5. Think of a real choice you made on a project of your own. Could you state the exact number or event that would make you reverse it? If not, what does that tell you?
Show hint
Try to name a specific number, not a feeling like "if it stopped working."
Show answer
Model answer: Most people can name the decision but not the number, which is exactly the gap this answer is pointing at: a real pick has a real threshold behind it.
Short answer, the number question
6. If Solveig had logged her kill signal the moment she made the call instead of four months later, would her answer to Tobias have changed? Why or why not?
Show hint
Think about what she actually had, versus what she had to go dig up.
Show answer
Model answer: The number wouldn't have changed, but the answer would have come in seconds instead of after four hours of digging, and it would never have been at risk of being lost entirely.
Before you close the answer
Why this works
Tests whether you understand that documenting a tradeoff is itself a tradeoff, between a format that's fast to write and one that's fast to trust, and whether you can name which cost you're choosing to hide.
Follow-up traps
"Isn't a structured record just as easy to fake as a story?" Response: harder, because a fake kill signal has to survive being cross-checked against real evidence, an eval log or a screenshot, while a fake story only has to sound plausible once.

"What if you genuinely can't remember the exact number after the fact?" Response: that's the whole argument for logging it in the moment. A number you can't reconstruct later is a number you never really had, only felt like you did.
If pressed
Civora's actual eligibility matcher runs a second, cheaper check before ever showing a match: it re-scores its own top suggestion against the grant's hard requirements, and anything that fails that second check gets flagged for manual review instead of being sent, the exact guardrail that caught the false-positive pattern behind Solveig's real kill signal.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more