ConceptIntermediateResponsible AI & Advanced Practice / Building an AI PM portfolio / #9
How do you show product judgement rather than technical execution in a portfolio?
PICK Owen Mackey built a diagnostic assistant for field technicians at Ferncrest Mechanical Services, an HVAC service company
Owen Mackey spent six years as an HVAC field technician before moving toward product work. His portfolio project is a diagnostic assistant for Ferncrest Mechanical Services, a mid-sized HVAC service company: it suggests likely causes for a unit's fault codes before a technician even opens the panel. Bridget Calloway hires AI PMs, and she's seen plenty of technically polished portfolios that never once show her how the candidate actually thinks.
The direct answer
Lead with the decision, not the code. Put a named alternative you rejected, and why, ahead of your model's accuracy number. A polished build with no visible reasoning behind it hides a judgment gap that only surfaces after a bad hire. A rough build with a clearly named trade-off shows judgment immediately, and a reviewer forgives the rough edges far more easily than the silence.
Do this, in order
Name the alternative you rejected, and why, before describing what you built.Why: this is the one thing that separates judgment from mere competent execution.
Put the judgment write-up ahead of the technical build in the reading order.Why: a reviewer forms their read on judgment before they ever test the code.
Own a wrong call you made, and what you changed because of it.Why: a portfolio with zero owned mistakes reads as untested, not as flawless.
Let the build stay rough where rough doesn't cost anything real.Why: polish spent on low-stakes parts of the build is polish not spent proving judgment.
Never let a high accuracy number stand in for an explained decision.Why: a number alone can't tell a reviewer whether the choice behind it was thoughtful or accidental.
Switch the emphasis back to execution only when the role is genuinely engineering-heavy.Why: for a hands-on ML engineering role, technical depth is the actual job, not a stand-in for it.
How to answer this, stage by stage
This isn't a portfolio-formatting question. It's asking whether you understand that a hiring panel is evaluating your thinking, and that clean code alone doesn't show them any of it.
Stage 1
Scope it to one real portfolio piece
Say it like this
"I'll answer this for one project inside a portfolio: a working build, plus whatever's written around it."
Why this works
Turns an abstract question about "showing judgment" into something concrete you can actually design.
Stage 2
Say your structure out loud
Say it like this
"I'll use PICK. Position, judgment first, always. Impact, who's misled by each option. Cost asymmetry, which gap is worse. Kill criteria, when execution should actually lead instead."
Why this works
Signals a deliberate choice about emphasis, not just a claim that judgment "matters."
Stage 3
State your position, before any reasoning
Say it like this
"My position: lead with the decision and the rejected alternative, every time, even if it means the build stays rougher than it could be."
Why this works
Commits to a clear stance instead of hedging into "both matter equally."
Stage 4
Name who's misled by each option
Say it like this
"A reviewer looking at only clean code can't tell if Owen chose his approach thoughtfully or just used the first thing that worked. A reviewer looking at only judgment prose with no code can't tell if he can actually ship."
Why this works
Shows both sides carry a real risk of misleading the reviewer, not just one obvious failure mode.
Stage 5
Name the cost asymmetry
Say it like this
"Rough execution is visible right away, a reviewer sees it and asks about it directly. A judgment gap hidden behind polished code is invisible, it doesn't get caught until a bad hire, weeks or months later."
Why this works
This is the heart of PICK: the hidden, expensive gap matters more than the cheap, visible one.
Stage 6
Prove it with the actual example
Say it like this
"Owen's write-up opens with: 'I considered ranking every fault code by frequency, but chose to rank by repair cost instead, since a rare but expensive fix matters more to a technician's day than a common cheap one.' That's one sentence, and it shows more than his whole model architecture section."
Why this works
This is the concrete, speakable proof of the answer, not just the theory behind it.
Stage 7
Give kill criteria, and close
Say it like this
"For a hands-on ML engineering role, technical depth genuinely is the job, and execution should lead. For an AI PM role, it's judgment first, execution as proof it holds up."
Why this works
Shows the position has a real limit, tied to the actual role being applied for.
Let's learn
Picture two portfolio pieces, built by two equally skilled people, that leave a reviewer with completely different impressions.
Before Owen rewrote his, the project page opened with architecture details: which model, which features, a validation accuracy of 91%. No sentence anywhere explained why he'd made a single one of his design choices.
Knowledge spark: what's a rejected alternative?
The other real option you considered and didn't choose, named specifically, along with the actual reason you passed on it. Not a vague "I explored a few approaches." A rejected alternative is the clearest, cheapest way to prove a decision was actually made, not stumbled into.
After the rewrite, the page opens with the fault-code ranking decision, names the frequency-based approach he rejected, and only then gets to the model and the accuracy number, now framed as proof the chosen approach actually works.
How many follow-up questions it takes to surface a real judgment gap, by portfolio style
A judgment-led portfolio surfaces the real signal almost immediately. An execution-led one takes four times as many questions to get there, if it gets there at all.
At its worst: a technically excellent candidate gets hired on the strength of clean code and a strong accuracy number, and three months in, it turns out every design choice was picked by trial and error, with no real reasoning behind any of it.
The cost asymmetry that matters here
Rough execution is visible immediately, a reviewer notices and can ask about it directly, cheaply. A judgment gap hidden behind polished execution is invisible, and it doesn't surface until a bad hire, months later, when it's far more expensive to discover.
What I would leave alone: the model architecture and accuracy number still belong on the page. They're just not the opening argument anymore. Once a reviewer has seen the reasoning, the technical detail becomes proof it actually holds up.
Clean code tells a reviewer you can build something. A named, rejected alternative tells them you decided what to build, on purpose.
The lesson: execution answers "can this person build it." Judgment answers "should this person be the one deciding what gets built." Only one of those is actually the job.
Now here is the same thing as a story
The short version above is what you'd say defending your own portfolio's structure. Read this one for how Owen actually found the fix.
Owen could stand in front of a rooftop unit throwing three fault codes at once and know, within a minute, which one was the real problem and which two were just noise from the same failure.
His first portfolio draft read like a technical report: model choice, feature engineering, validation split, a clean 91% accuracy number sitting at the top like a trophy.
One kind of gap gets caught in the first five minutes. The other one hides behind good-looking work until it doesn't.
He got through three interviews on the strength of that draft, and each time the same thing happened: a technical question he could answer fine, followed by silence when asked why he'd made a specific design choice at all.
The reviewer's real decision happens at the second stop, not the first. Owen's old draft had almost nothing waiting there.
Bridget, reviewing his draft as a favor, put it plainly: "I believe you can build this. I have no idea if you decided to build it this way for a reason."
Owen's draft sat in the corner with the most polish and the least judgment shown. That corner is more common than it should be.
He spent a weekend rewriting the page from the decision outward: the fault-code ranking choice first, the rejected frequency-based approach named plainly, and one wrong call owned honestly, a threshold he'd set too aggressively at first and had to loosen after testing.
Four parts, and none of them require the code underneath to be any cleaner than it already was.
The decision comes first now. The build comes after, as evidence the decision actually worked.
Judgment and execution both belong on the page. They're just answering two completely different questions.
The old draft asked Bridget to infer judgment from clean code, which she couldn't do reliably. The new one hands her the reasoning directly, and lets the code sit underneath it as proof, not as the argument itself.
I built the technical section first because it was the part I felt most confident about, and I assumed the reasoning would come through on its own if the work was good enough. Hearing Bridget say she had no idea if I'd decided anything on purpose is what showed me good work was never going to speak for the decisions behind it.
PICK, on judgment versus executionNot a style preference. A question about which gap actually costs more.
P
Position. Stated before any reasoning.
Judgment first, execution second, even if that means the build stays rougher than it could be.
The hardest step, and the direct answer to the whole question.
I
Impact. Who's misled by each option.
Execution alone hides whether a choice was thoughtful. Judgment alone with no build hides whether the person can actually ship.
Names a real risk on both sides, not just the obvious one.
C
Cost asymmetry. Which gap is worse.
Rough execution is visible and cheap, caught in the first five minutes. A judgment gap behind polish is hidden and expensive, caught only after a bad hire.
The heart of the whole answer: optimize against the hidden, expensive gap.
K
Kill criteria. When this flips.
For a hands-on ML engineering role, technical depth genuinely is the job, and execution should lead the page instead.
Shows the position has a real, role-specific limit, not a universal rule.
Where PICK flips: role type versus recommended emphasis
The recommendation flips gradually, not all at once, and it fully crosses over only once a role is genuinely hands-on engineering work.
The recap, one line per letter: position is judgment before execution, always, unless the role says otherwise; impact is that either extreme misleads a reviewer in a different way; cost asymmetry is the hidden, expensive judgment gap beating the cheap, visible execution gap; and kill criteria is a genuinely engineering-heavy role flipping the emphasis back.
And if you want to be sure it really works, try it somewhere elseSame four letters, a textile quality-inspection project instead of HVAC diagnostics. A different candidate, and this time the rough edge was in the judgment write-up itself.
Tobias Kemp built a fabric-defect flagging tool for a small textile mill's quality-control line. Applied to PICK: position is still judgment first. Impact: a reviewer seeing only Tobias's clean defect-detection code can't tell if he understood which defects actually mattered to a buyer versus which were merely visible. Cost asymmetry: same shape, a rough interface gets forgiven quickly, but Tobias's first write-up had the opposite problem, confident judgment language with no specific rejected alternative named, which read as vague rather than reasoned. Kill criteria: for this specific role, a hands-on textile-manufacturing engineering team, the panel actually wanted more technical depth than Tobias initially included, since defect classification accuracy genuinely mattered to their day-to-day process.
The same four parts apply, but here the missing piece wasn't execution polish. It was a specific enough rejected alternative.
Swap the trigger and it still runs.
Speed: an interviewer wants your answer in ten seconds. Say "lead with the decision you made, not the code that carried it out," and stop.
Cost: you don't have time to polish the build at all. Ship it rough and spend the saved time writing the judgment call clearly instead.
The role shifts, for real: if a role turns out to be more engineering-heavy than expected, the position doesn't disappear, it just moves down the priority list, behind a real technical deep-dive.
Where people run it wrong.
They treat a high accuracy number as if it speaks for the judgment behind it, when it doesn't.
They write about judgment in the abstract, "I carefully considered trade-offs," without ever naming the specific alternative they rejected.
They assume more polish always helps, when polish spent on low-stakes parts of a build is polish not spent proving judgment.
How to use it live. When asked how to show judgment over execution, answer with the concrete move itself: name one alternative you rejected, and why, before describing anything you built. That single sentence does more work than any amount of code.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits "how do you show product judgment rather than technical execution"?
Tap to flip
ANSWER
PICK: position, impact, cost asymmetry, kill criteria. Position lands on judgment first, execution as supporting proof.
2 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Owen Mackey, a former HVAC technician building a diagnostic assistant for Ferncrest Mechanical Services, and Bridget Calloway, the hiring manager reviewing it.
3 · THE HABIT
What habit did Owen have to drop, once he found the real fix?
Tap to flip
ANSWER
Leading with technical architecture and an accuracy number, assuming good work would make the reasoning behind it obvious on its own.
4 · THE ASYMMETRY
Which is worse: rough execution, or a judgment gap hidden behind polish?
Tap to flip
ANSWER
A judgment gap hidden behind polish. It's invisible and doesn't surface until a bad hire, while rough execution gets noticed and questioned right away.
5 · THE MOVE
What's the single concrete move this answer recommends?
Tap to flip
ANSWER
Name the specific alternative you rejected, and why, before describing what you actually built.
6 · THE NUMBER
Fill in the blank: an execution-led portfolio takes about ___ follow-up questions on average to reveal a real judgment gap, versus 1.1 for a judgment-led one.
Tap to flip
ANSWER
4.2 questions. That's roughly four times as many, and it may still never fully surface within a single interview loop.
7 · THE REPLAY
Same three interviews, redesigned portfolio page. What changes?
Tap to flip
ANSWER
Bridget reads the rejected alternative first, understands the reasoning within the first few sentences, and treats the accuracy number as confirmation rather than the whole argument.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different candidate. Who, and what was the flipped kill criterion there?
Tap to flip
ANSWER
Tobias Kemp's textile defect-flagging tool. There, the hiring panel for a hands-on engineering role genuinely wanted more technical depth than his judgment-first write-up initially gave them.
Check yourself Score: 0 / 0
Multiple choice
1. According to this answer, what's the single strongest move for showing judgment in a portfolio?
A. Achieving the highest possible accuracy number.
B. Writing cleaner, better-commented code.
C. Naming a specific alternative you rejected, and why, before describing the build.
D. Adding more features to the demo.
Show hint
Look at the priority list and "how to use it live."
Show answer
C. A named, rejected alternative is the cheapest and clearest way to prove a real decision was made, not stumbled into.
True or false
2. True or false: this answer says technical execution should never lead a portfolio page, under any circumstances.
True
False
Show hint
Look at the K, kill criteria, step.
Show answer
False. For a hands-on ML engineering role, technical depth genuinely is the job, and execution should lead instead.
Fill in the blank
3. Fill in the blank: an execution-led portfolio takes about ___ follow-up questions on average to reveal whether real judgment exists.
Show hint
Look at the grouped bar chart comparing portfolio styles.
Show answer
4.2 questions. A judgment-led portfolio needs only about 1.1 questions on average to surface the same signal.
Short answer, name the reversal
4. What decision would Owen take back from his first portfolio draft, and why did it seem reasonable at the time?
Show hint
Look at the story's description of his first draft and Bridget's feedback.
Show answer
Model answer: Leading with architecture and an accuracy number, assuming strong technical work would make his reasoning obvious without having to state it directly.
Short answer, where it wouldn't matter
5. Name a role where leading with execution over judgment would actually be the right call.
Show hint
Think about what the actual job requires day to day.
Show answer
Model answer: A hands-on ML engineering role, where technical depth and implementation quality are the literal job, not a stand-in for something else.
Short answer, apply it yourself
6. Think of something you've built or decided recently. What's one specific alternative you rejected, and why, that you've never actually said out loud to anyone?
Show hint
Look for a moment where you picked one option over a real, specific other one, not just "considered a few approaches."
Show answer
Model answer: Most people can name a specific alternative once pushed, they just haven't thought to say it out loud as part of describing their work.
Before you close the answer
Why this works
Tests whether you understand that a hiring panel is trying to evaluate your thinking, not your code quality, and whether you'd know how to make that thinking visible instead of assuming good work speaks for itself.
Follow-up traps
"Isn't naming a rejected alternative just a writing trick?" Response: no, since it has to be a real option you genuinely considered, and a reviewer can always ask a follow-up question that exposes whether it was real or invented after the fact.
"What if the technical work really is the most impressive part?" Response: then say so explicitly, and lead with the reasoning for why that technical depth was the right call, rather than letting the number speak alone.
If pressed
Owen's rejected frequency-based ranking would have surfaced common, cheap fixes first, and he can point to two real fault-code logs where that approach would have buried an expensive compressor failure behind a dozen routine filter alerts.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.