ConceptAdvancedDesigning for Uncertainty & Trust / Onboarding users to probabilistic products / #10
Explain how onboarding differs for users who are skeptical versus enthusiastic.
PICK the product is LedeCraft, an AI headline and summary drafting assistant for a newsroom
The Solborne Ledger rolled out LedeCraft to its whole newsroom this spring. Jarrah Osei has copyedited there for fourteen years and trusts almost nothing on the first try. Wren Calloway joined six months ago and had LedeCraft's headline drafts loaded before her onboarding email finished loading.
The direct answer
Skeptical and enthusiastic users don't fail the same way, so onboarding shouldn't treat them the same way. Give the skeptic a narrow, checkable proof up front, one thing they can verify themselves in under a minute. Give the enthusiast a forced pause before their first publish, because their real risk isn't refusing the tool, it's trusting it too fast and never looking back.
Do this, in order
Design against the hidden, expensive error, not the loud, cheap one.Why: a skeptic quitting shows up on a dashboard immediately. A wrong headline slipping past an enthusiast shows up days later, in a correction.
Give the skeptic one small, checkable win, not a sales pitch.Why: a claim doesn't earn trust from someone who doesn't trust claims. A verified detail does.
Give the enthusiast one required friction step before their first publish.Why: their failure mode is speed, not hesitation, so the fix has to slow the one moment that matters.
Detect which kind of user this is from behavior, not from a survey question.Why: people rarely self-report as either skeptical or overconfident.
Keep the underlying model and data exactly the same for both paths.Why: only the first ten minutes of introduction should differ, not the tool itself.
How to answer this, stage by stageSix moves, in the order you'd actually say them.
Stage 1
Scope it to one real newsroom
Say it like this
"I'll answer this for LedeCraft, a headline and summary drafting tool, at a newsroom where it just became mandatory for the whole desk."
Why this works
Keeps "skeptical versus enthusiastic" from staying an abstract personality question.
Stage 2
Say your structure out loud
Say it like this
"I'll use PICK. Position, my call up front. Impact, who feels which error. Cost asymmetry, which one to optimize against. Kill criteria, what would change my mind."
Why this works
Tells the interviewer this is a real tradeoff being resolved, not a list of nice-to-haves.
Stage 3
Reframe the question
Say it like this
"This isn't really about personality. It's about which of two failure modes each person is walking toward: quitting too early, or trusting too fast."
Why this works
Moves the answer past "be more welcoming to skeptics" into an actual design problem.
Stage 4
Give the one decision
Say it like this
"The skeptic gets a small, checkable proof on their first story. The enthusiast gets a required compare-to-source tap before their first publish. Same model underneath, two different first ten minutes."
Why this works
This is the position. Stated once, plainly, before any story.
Stage 5
Name the cost asymmetry
Say it like this
"If the skeptic quits, we see it on a usage chart the same day. If the enthusiast trusts a wrong headline, we might not see it until a reader catches it and we run a correction."
Why this works
This is the heart of PICK: naming which error is hidden and expensive, so you know which one to design against.
Stage 6
Close on the one line
Say it like this
"So the two onboarding paths aren't about personality types. They're two different insurance policies against two different kinds of mistake."
Why this works
Restates the position and its reason in one breath.
Let's learn
LedeCraft reads a filed story and drafts a headline and a two-line summary in about eight seconds, pulling quotes and figures straight from the copy.
Before it, a reporter wrote their own headline and summary by hand, usually five to ten minutes of work on deadline, checking their own memory of the story against what they'd actually written.
Knowledge spark: what's a "compare-to-source" view?
A small panel that sits the AI's draft next to the actual sentence in the story it pulled from. One tap shows where a quote or number came from, so nobody has to trust a claim, they can just look.
The Ledger rolled LedeCraft out with one onboarding flow: a friendly two-minute tour, the same for every reporter and editor on staff, regardless of how they felt about it walking in.
Same tour, same tool, and two people who were never going to fail the same way.
And here's the turn: giving both of them the same reassuring tour doesn't fail because it's unkind to one of them. It fails because it's aimed at the wrong risk for each.
Both panels are real costs. They are not the same size, and they don't show up on the same day.
At its worst: a skeptic like Jarrah opens LedeCraft once, sees a generic pitch instead of proof, and never opens it again, quietly working around it for the rest of the year. Or an enthusiast like Wren publishes a headline built on a quote LedeCraft slightly misattributed, and nobody catches it until a reader emails in.
The choice I would take back
We wrote one onboarding tone for everyone: upbeat, fast, reassuring, because our first hundred users were enthusiastic early adopters who chose to try LedeCraft themselves. That made sense while everyone using it had opted in. It stopped making sense the day the newsroom made it mandatory for the whole desk, including people who never asked for it and people who trusted it before they'd earned any reason to.
What I would leave alone: the model's actual drafting quality doesn't need two versions. The same headline, the same summary, the same accuracy, for everyone. Only the first ten minutes of how it's introduced should differ.
We weren't onboarding two personalities. We were insuring against two different mistakes, and we'd only built one policy.
The lesson: "onboarding" isn't one flow with a friendly wrapper. It's a bet about which way a specific person is most likely to fail, and the bet has to be made on purpose.
Now here is the same thing as a storyThe short version is above. Read this for how the two paths actually got built.
Jarrah can spot a misquote from across the newsroom. Fourteen years of reading other people's copy does that to a person.
His first week with LedeCraft went the way most tools go for him: he opened it once, watched it draft a plausible-looking headline, and closed the tab. He didn't come back for four days.
None of these are complaints. They're the exact behavior a proof-first onboarding is built to meet.
Then a colleague, half-joking, said "you know it's usually right, right?" over coffee. Jarrah didn't answer. He went back to his desk and, this time, opened the compare-to-source panel on a story LedeCraft had drafted a headline for, and checked the one pulled quote against the actual transcript.
The remark didn't convince him. The check did. That's the whole design principle, in three weeks.
It matched, word for word. He checked two more that week. They matched too. By week three he wasn't reading every draft anymore, he was spot-checking the ones with a direct quote in the headline, which is exactly the habit a proof-first design is meant to build.
A user who's assigned rather than self-selected gets routed the same way a cautious one does. Neither chose this, and neither has earned trust yet.
Wren's story ran the opposite direction. Her first week, she accepted eleven of LedeCraft's twelve suggested headlines without opening a single source story, because they all read cleanly and she was thrilled to file three extra stories that week.
This is the piece that runs in the opposite direction from Jarrah's proof: not offered when asked for, required before she can move on.
The near miss came on headline nine: LedeCraft had drawn a percentage from a paragraph two stories back in a linked document, not the story Wren was filing. She caught it by luck, a stray number that looked slightly too large, and re-read the source herself for the first time all week.
With the required compare-to-source tap already built into her onboarding path, that same near miss becomes routine instead of lucky: she can't publish until the panel opens, so the mismatched number gets caught the same way every time, not just the one time she happened to notice.
I built one onboarding flow because it was simpler to maintain and felt fairer, the same welcome for everyone. It took watching a near miss that easily could have run in the paper to see that "fair" and "aimed at the same risk for everyone" are not the same thing.
PICK, in one screenFour letters. The third one is the whole argument.
P
Position. The call, before any reasoning.
Skeptics get a small, checkable proof. Enthusiasts get a required pause before their first publish.
Commits to an answer instead of describing both sides forever.
I
Impact. Who feels each error.
The skeptic who quits feels it as wasted time they never got back. The reader who catches a wrong headline feels it as a broken trust in the paper itself.
Names both sides in real, human terms, not just a metric.
C
Cost asymmetry. The heart of it.
A skeptic quitting is visible the same day on a usage dashboard. A wrong headline slipping past an enthusiast is invisible until a reader or a correction desk catches it, days later.
The hardest step. Optimize the design against the hidden, expensive one.
K
Kill criteria. What would flip the call.
If a real study showed skeptics were leaving over friction, not distrust, the proof-first path would need to shrink, not grow. If enthusiasts' correction rate never rose above baseline, the required pause could become optional.
Separates a confident answer from a stubborn one.
Week-1 outcomes, generic onboarding vs. split-path onboarding
Both risks shrink once the onboarding path matches the person, not just one of them.
The recap, one line per letter: position is the two different paths, impact is naming who feels each error and how, cost asymmetry is the hidden, delayed cost of over-trust against the visible, immediate cost of abandonment, and kill criteria is the evidence that would rebalance either path.
And if you want to be sure it really works, try it somewhere elseSame four letters, a translation agency instead of a newsroom.
Meridian Language Partners uses an AI drafting tool for first-pass translations, which human translators then review. Ingrid Fossum has translated technical manuals there for eleven years and treats every AI draft as a rough starting point to be argued with, not trusted.
Mapped onto PICK: position is the same split, a checkable proof for skeptical veteran translators, a required second-pass flag for eager new hires who accept drafts fast. Impact is a veteran who quits the tool wasting nothing but their own time, versus a new hire whose unflagged mistranslation ships in a safety manual. Cost asymmetry favors the new hire's risk overwhelmingly, since a mistranslated safety instruction is a far larger cost than a slower veteran adoption curve. Kill criteria: if data showed veteran translators were actually the ones making unflagged errors, not new hires, the required-flag design would need to apply to both groups equally.
Self-reported trust in AI drafts, weeks 1 to 6
Trust doesn't rise from a friendlier tour. It rises from checks that keep coming back true, week after week.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "a proof for the skeptic, a required pause for the enthusiast," and stop.
Cost: no budget to build two separate onboarding flows this quarter. Say so honestly, and start with one cheap signal, whether someone opens the compare-to-source panel unprompted in their first session, and route from there.
The model gets better, for real: if LedeCraft's accuracy improves across the board, the enthusiast's risk doesn't disappear, it gets quieter and more dangerous, since a rarer error is an error nobody's watching for anymore.
Where people run it wrong.
They build one onboarding tone for everyone and call it fairness, when it's really just one bet on one failure mode.
They assume skepticism is the harder problem to solve, when an enthusiast's silent over-trust is usually the more expensive one.
They ask users to self-report their own trust level, which people are bad at doing honestly.
How to use it live. When someone asks how onboarding differs by disposition, ask yourself one question first: which of these two people is more likely to cost you something you won't see coming. Design against that one first.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits "explain how onboarding differs for skeptical versus enthusiastic users"?
Tap to flip
ANSWER
PICK: position, impact, cost asymmetry, kill criteria. It's a tradeoff question in disguise, between two failure modes.
2 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Jarrah Osei, a fourteen-year copy editor who trusts nothing at first, and Wren Calloway, a reporter six months in who trusted LedeCraft immediately.
3 · THE POSITION
What's the actual design commitment, in one line?
Tap to flip
ANSWER
Skeptics get a small, checkable proof. Enthusiasts get a required compare-to-source tap before their first publish.
4 · THE ASYMMETRY
Which error is hidden and expensive, and which is visible and cheap?
Tap to flip
ANSWER
A skeptic quitting is visible the same day on a dashboard. An enthusiast's unchecked wrong headline is hidden until a reader or correction desk catches it.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Writing one upbeat onboarding tone for everyone, which made sense while every user had opted in, and stopped making sense once the tool became mandatory for people who hadn't.
6 · THE NUMBER
Fill in the blank: under generic onboarding, skeptic abandonment ran ___ percent, versus 9 percent under split-path onboarding.
Tap to flip
ANSWER
38 percent. The enthusiast's uncaught error rate dropped from 6.2 to 1.1 per 100 headlines over the same change.
7 · THE REPLAY
Same near-miss headline, redesigned onboarding. What changes for Wren?
Tap to flip
ANSWER
The compare-to-source panel is required before she can publish, so the mismatched figure gets caught every time, not just the one time she happened to notice.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and how does the cost asymmetry shift?
Tap to flip
ANSWER
Meridian Language Partners' translation tool. There, the asymmetry tilts even harder toward the enthusiast's risk, since an unflagged mistranslation can ship inside a safety manual.
Check yourself Score: 0 / 0
True or false
1. True or false: this answer recommends giving every new LedeCraft user the exact same onboarding tour.
True
False
Show hint
Look at the position step and the decision tree diagram.
Show answer
False. Skeptics get a proof-first path, enthusiasts get a friction-first path, routed by behavior, not by a survey question.
Multiple choice
2. Why does the answer optimize the design against the enthusiast's error rather than the skeptic's abandonment?
A. Because skeptics never actually quit using tools.
B. Because enthusiasts are a bigger share of the newsroom.
C. Because the enthusiast's error is hidden and can cost a public correction, while a skeptic quitting is visible right away.
D. Because the model performs worse for enthusiastic users.
Show hint
Look at the cost asymmetry step.
Show answer
C. A hidden, expensive error is worse to leave unaddressed than a visible, cheap one, even if the cheap one feels more urgent day to day.
Fill in the blank
3. Fill in the blank: the required feature in the enthusiast's path is a ___ panel, shown before the first publish.
Show hint
Look at Wren's story and the labeled-parts diagram.
Show answer
Compare-to-source. It shows the original quote next to the AI's draft, so a mismatch is visible without anyone needing to trust a claim.
Short answer, where it wouldn't matter
4. Name something about LedeCraft that stays exactly the same for both the skeptic and the enthusiast.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: The underlying model, its drafting quality, and its accuracy. Only the first ten minutes of introduction differ, not the tool itself.
Short answer, name the reversal
5. What old decision does this answer take back, and why did it make sense when it was made?
Show hint
Look at "the choice I would take back."
Show answer
Model answer: One upbeat onboarding tone for everyone. It made sense while every early user had opted in voluntarily, and broke once the tool became mandatory for people who hadn't chosen it.
Short answer, apply it yourself
6. Pick a tool you use yourself. Are you more the skeptic or the enthusiast with it, and what would a good onboarding have done differently for you?
Show hint
Think about a tool you either avoided at first or trusted immediately, and what would have changed that.
Show answer
Model answer: Many people describe trusting a new finance or health app immediately because the interface looked polished, then getting burned by one wrong number, which is exactly the enthusiast's risk this answer designs against.
Before you close the answer
Why this works
Tests whether you'll treat "onboarding" as one friendly wrapper or actually design two different defenses against two different, real mistakes, and whether you can name which one costs more when nobody's watching.
Follow-up traps
"Isn't forcing a compare-to-source tap just annoying friction for your best, fastest users?" Response: yes, a little, and that's the trade being made on purpose, since those are exactly the users most likely to skip a check they'd otherwise never think to make.
"How do you know someone's a skeptic or an enthusiast without asking them directly?" Response: behavior in the first session, whether they open the source panel unprompted or accept drafts without opening it, is a more honest signal than a self-reported survey answer.
If pressed
The real routing signal at the Ledger used time-to-first-accept: anyone who accepted a suggested headline in under four seconds on their first try got routed to the friction-first path automatically, no survey question involved.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
