ConceptIntermediateModel Fluency & the AI PM Role / What changes when the product is probabilistic / #14
Describe the relationship between sampling temperature and product predictability.
BOUND · AI drafting tools for novelists
Draftkiln is Inkbramble Labs' writing engine for novelists stuck mid-scene: feed it your last paragraph, and it writes what could happen next. Four features run on that same engine, and for months they all shared one temperature setting, 0.3, because it was the number already trusted for the feature that had to stay exact. Vespera Fenwrick owns that number. When a beta author called the tool "autocomplete in a trench coat," Vespera had to work out what "predictable" actually costs, and where creativity is worth paying for it.
The direct answer
Temperature controls how narrowly the model samples its own list of likely next words: near zero it almost always repeats the single most probable one, so the same prompt returns nearly the same output every time, and turned up, words that were only slightly less likely get a real chance, so twenty reruns can head in different directions. There's no one right setting, only a defensible range per feature: low wherever the job needs the same answer twice, higher wherever a different answer each time is the whole point.
Do this, in order
Set temperature low wherever the job needs the same answer twice, and higher wherever a different answer each time is the value.Why: it's the one setting that swings the same-beat rate from 90 percent to 25 percent in Draftkiln's own reruns, more than anything else the team changed.
Break down what the number is actually doing before recommending a setting.Why: skip the mechanism, how the model reshapes its own odds, and "predictable" turns into a feeling nobody can defend in a review.
Own the reroll cost that comes with turning it up, not just the creativity.Why: silent rerolls climbed from about 4 percent of outputs at 0.3 to about 35 percent at 0.8, a real cost per accepted paragraph.
Give each feature its own range, not one shared number.Why: 0.7 to 0.9 for scene continuation against 0.0 to 0.15 for fixing continuity errors; one borrowed default was the actual bug.
Check the range against what a real author can notice, not just what the rerun count shows.Why: a blind read test found readers called 0.3-versus-0.75 a genuinely different direction 83 percent of the time, far past the 10 percent noise floor from two same-setting reruns.
Keep the raw number out of the author-facing screen.Why: nobody drafting a novel should have to learn what a sampling parameter is; translate it into a labeled choice instead.
How to answer this, stage by stage
Nobody is grading whether you can say "we'd tune that carefully." They're grading whether you can turn a soft word like predictable into a mechanism, a counted number, and a range you'd actually defend to a person who noticed the difference.
1
Ground it in one real product decision
Say it like this
"Let's make this concrete. Draftkiln is Inkbramble Labs' scene-continuation engine for novelists. Four features run on it, and for months they all used the same temperature, 0.3, because that was the number already trusted for the feature that has to stay exact. Vespera Fenwrick owns that setting, and a beta author's complaint is what made her actually go measure it."
Why this works
One real product and one real number keeps the whole answer checkable, instead of a lecture about sampling in the abstract.
2
Say what temperature is actually doing, mechanically
Say it like this
"Temperature isn't a vibe dial. Every next word starts as a list of candidates with scores, and temperature decides how much of that list gets a real shot. Near zero, it squeezes almost all the odds onto the single top candidate. Push it up, and the second and third choices start getting real odds too."
Why this works
This is the load-bearing idea. Skip the mechanism and "predictable" stays a guess.
3
State the range you'd recommend, before the arithmetic
Say it like this
"Here's where I'd land: keep continuity fixes and character renaming near zero, 0 to 0.15, because those need the same right answer twice. Push scene continuation up to 0.7 to 0.9, because a stuck writer wants a different idea each time, not the same paragraph back."
Why this works
Saying the decision before the numbers is the direct answer, stated plainly, before anyone mistakes this for a lecture on sampling theory.
4
Own the numbers, and where each one came from
Say it like this
"We reran the same stuck-scene prompt twenty times at four settings. At 0.3, eighteen of twenty landed on the same next beat. At 0.8, only five did. That's not a guess, it's a count, from an actual rerun log, not a spec sheet."
Why this works
A number with a source survives a follow-up question. A number with no source is just a confident guess.
5
Sanity-check it against what a person actually feels
Say it like this
"Forty beta authors got shown two continuations of the same paragraph, one at 0.3, one at 0.75. Eighty-three percent called it a genuinely different direction. When we showed two reruns both at 0.3, only ten percent said that, which is just noise. So 0.75 isn't just a bigger number, it's a difference a real reader notices."
Why this works
A range that never gets checked against a real person is just an internal metric wearing a decision's clothes.
6
Name the one decision that moves it most
Say it like this
"If you push me on what actually swings this, it's the temperature setting itself, not the prompt, not the model version. Doubling the prompt barely moved the match rate. Moving temperature from 0.3 to 0.8 dropped it sixty-five points. That's the lever."
Why this works
Naming the single most load-bearing variable is what separates a real estimate from a list of things that might matter.
7
Close on the decision, in one breath
Say it like this
"So: temperature sets how many next-words get a real chance. Keep it low wherever the same answer twice is the job, raise it wherever a different answer each time is the value, and check both ends against what a real reader notices, not just what the rerun count says."
Why this works
Restating the decision plainly means whoever's listening leaves with the answer, not just the story.
Let's learn
What happens when the same setting gets reused for a job it was never actually tested against?
Say we build a tool that reads the last paragraph a novelist wrote and writes a few sentences of what could happen next, so a stuck writer has something to react to instead of a blank page.
Before this feature existed, a stuck writer in early testing sat with an empty cursor for about 24 minutes, on average, before typing a new sentence. With one AI-written continuation to react to, even a rough one, that dropped to about 6 minutes. Reacting to something on the page is easier than making something out of nothing.
Knowledge spark: what's greedy decoding?
Always take the single most likely next word, no randomness at all. Temperature at zero is nearly the same thing: the top word almost always wins anyway, so the output barely changes rerun to rerun.
The launch config. One number, copied into four config files because it had already earned trust on one of them.
The team launched with one shared setting across every feature built on the engine: temperature 0.3, the number already trusted for the feature that fixes timeline mistakes without changing the plot. It felt safe to reuse a number that had already proven itself instead of picking four new ones. For the continuity fixer and the rename tool, 0.3 was exactly right. For "continue the scene," it was quietly wrong. Rerun the same stuck-scene prompt twenty times at 0.3, and eighteen of the twenty come back on the same next beat.
Twenty reruns of one prompt, matched vs diverged, by temperature
Matched the same next beatDiverged to a different beat
At 0.3, eighteen of twenty reruns land on the same next beat. At 1.1, only two do. One shared number swings from almost no variety to almost all of it.
Here's the turn. The extra sameness isn't really the problem, on its own. A grammatically fine, in-character paragraph isn't a mistake. The real problem is what a stuck writer does after seeing it twice: they stop rerunning it, and then they stop opening the feature at all, because they've learned it's going to say roughly the same thing either way.
The word predictable wasn't wrong. It just hadn't decided which job it was talking about.
The same setting can fail in two opposite directions. Turn temperature too low and you get the machine. Turn it too high, with nothing checking it, and you get the dog.
What it costs at its worst: a feature built to help someone get unstuck quietly becomes a feature nobody trusts to actually help, and it never shows up as a bug report, because nobody files a ticket over a tool being boring. It just shows up months later as a usage chart that's been sliding for weeks.
The decision that mattered
Draftkiln's launch plan gave every new feature the continuity fixer's already-trusted temperature, 0.3, because reusing a proven number felt safer than picking four new ones. That was right for a task that needs the same right answer twice. It was the wrong number for a task whose whole job is handing someone an idea they didn't already have.
What I would leave alone: the continuity fixer and the character-rename tool. Both should stay near that same low setting on purpose. Getting a different "fix" for the identical mistake every time you run it would be its own kind of broken.
The lesson: "predictable" isn't a company-wide setting. It's a question you have to ask per feature, and answering it takes an actual rerun count, not whichever number already felt safe.
Now here is the same thing as a story
Skip this part under interview time pressure. Read it when you've got ninety more seconds and want to feel why one borrowed number broke a feature nobody had actually tested it against.
Vespera Fenwrick owns every setting on Draftkiln's writing engine at Inkbramble Labs. Ask her what temperature a given feature runs at and she can tell you before you finish the question, because she's the one who set every single one of them.
This is the test Vespera ran after the complaint landed. Same paragraph, same prompt, twenty separate tries.
Draftkiln shipped four features off the same engine in one sprint: fix a continuity error without changing the plot, rename a character consistently through a whole manuscript, continue a stuck scene, and fork an ending into three different directions. To keep the sprint simple, Vespera copied one number into all four config files, 0.3, the setting the continuity fixer's own spec already called safe. It shipped clean. Nobody flagged it in review.
For the continuity fixer and the rename tool, 0.3 was exactly right from day one, same mistake in, same fix out, every time, and beta testers loved that they never had to double-check the tool's own consistency. Early reviews for scene continuation were fine too. Most stuck writers just wanted something, anything, to push against.
In week one, writers reran a stuck scene two or three times, hoping for something different to react to. By week two, most reran it once. By week three, a few stopped opening it before even reading what it wrote, because they'd already learned roughly what it would say.
Then, in the beta Discord, an author named Enid Ashquist posted a screenshot: five reruns of the same paragraph, five versions of a character opening the same door. "Is this even generating anything," she wrote, "or is it just autocomplete in a trench coat?"
Vespera pulled Enid's exact prompt and reran it twenty times at the shipped 0.3 setting. Eighteen of the twenty landed on the same next beat as Enid's first one. Not eighteen similar paragraphs. Eighteen that opened the same door.
The real cost was bigger than one bad screenshot. Usage on "continue the scene" had been sliding for three weeks before Enid's post, quietly, with zero support tickets, because nobody writes in to complain that a feature is boring. It had dropped by something like 40 percent, and nobody had noticed, because the dashboard everyone watched was uptime and error rate, not whether writers still bothered to press the button twice.
We didn't lose eighteen matching paragraphs. We lost the one thing the feature was for: an idea the writer didn't already have.
The choice Vespera would take back happened in a fifteen-minute planning call, three sprints earlier. Someone asked what temperature the new features should ship at. Someone else said, "just use what the continuity fixer uses, it's already proven," and the room moved on to the next item. Nobody asked whether "proven" for one job meant anything for the other three.
Vespera reran Enid's exact prompt again, this time at 0.8, the number "continue the scene" runs at now. Five of twenty landed on the same beat, not eighteen. And in a blind read test, forty beta authors shown one paragraph at 0.3 and one at 0.75 called it a genuinely different direction 83 percent of the time.
One design hands a stuck writer the same door, opened the same way, eighteen times out of twenty. The other hands them five doors and fifteen different rooms.
What I would tell myself, back in that fifteen-minute call: "proven" was doing a lot of work for a number we'd only ever tested on one job.
Running BOUND on a slider nobody in the room had actually tested
Not a way to make "predictable" sound more technical. BOUND is what makes you show the mechanism, count the rerun, give the number a defensible width, and check it against a person who'd actually notice.
BBreak it down. What is temperature actually changing?
Every next word starts as a list of candidates with raw scores. Temperature reshapes those scores into odds before the model draws one. Near zero, nearly all the odds land on the single top candidate. Above about 0.6, second and third choices start getting real odds too, and a scene continuation strings together roughly 150 of these picks in a row, so the effect compounds fast across a whole paragraph.
Say the mechanism before naming a setting, or "predictable" stays a feeling nobody can defend.
A funnel near zero, a scale once it's turned up. Same list of candidate words, a different amount of it actually gets a chance.
OOwn the numbers. Where did each one actually come from?
Twenty reruns of one stuck-scene prompt, at four settings: 0.3, 0.5, 0.8, 1.1. Twenty isn't a big sample, it's an assumption, chosen because it's enough to see a clear pattern without burning a day of compute chasing a tighter number nobody was going to act on differently. "Same beat" was scored by a person reading the first two sentences of each output and marking whether the same thing happened next, not an exact-text match, because two paragraphs can use different words for the same event. And the close-call count, about 15 genuine near-ties in a 150-word continuation, came from manually marking candidates within ten points of the top pick across ten sample outputs, a real count on a small sample, not a company-wide constant.
Owning a number means saying where it came from and how much that source should be trusted, not just stating a figure that sounds right.
Knowledge spark: what's a close call?
Two or more next-words sitting almost equally likely. Which one "wins" comes down to the roll of the sampler, not a clear best answer, and a continuation with more close calls in it is one that can wander further from any earlier rerun.
UUse a range. Which one, for which job?
There's no single correct temperature, only a defensible band per feature. Fix a continuity error or rename a character: 0 to 0.15, because the job needs the same right answer twice. Continue a stuck scene: 0.7 to 0.9, because the whole value is a different idea each time. Fork an ending into three drafts: 1.0 to 1.2, because that feature's entire job is maximum spread. A single shared number, whatever it was, was always going to be wrong for at least two of these four jobs.
This is the direct answer to the question, run through the actual product: predictability isn't one setting, it's four ranges, each earned by what that feature is actually for.
One list, four different numbers. The fix for the borrowed default was never a single better number, it was four honest ones.
NNail the sanity check. Would a real person even notice?
Forty beta authors read two continuations of the same stalled paragraph blind, one from 0.3, one from 0.75. Eighty-three percent called it a genuinely different direction for the scene. As a control, the same forty readers got two separate reruns both at 0.3, and only ten percent called those different, which is just noise from how people read. So the 0.3-to-0.75 jump isn't a rounding difference dressed up as a range, it's a gap most readers actually feel.
The hardest step, and the one that gets skipped: a range that never gets checked against a real reader is just an internal number wearing a decision's clothes.
Repeat rate as temperature climbs, with each feature's shipped range
Repeat rate, out of 20 rerunsContinuity + rename rangeContinue the scene range
The burgundy band sits where the curve is nearly flat at the top, exactly what a same-answer-twice job needs. The teal band sits on the steep part of the curve, where turning the dial actually buys a different idea.
DDirection. Which one assumption moves the answer most?
Not the sample size, not the model version, not the prompt length. Doubling the prompt from 200 to 400 words moved the match rate at a fixed 0.8 from 25 percent to about 30, barely anything. Swapping the model's underlying version moved it maybe three points either way. Moving temperature itself from 0.3 to 0.8 dropped the match rate 65 points, more than every other change combined. If someone's only got budget to get one thing right, it's this setting, not the sample size, not the model choice.
Naming the single most load-bearing lever, and saying why it beats the other candidates, is what a real estimate does that a guess dressed up in numbers doesn't.
Temperature sits exactly where a good estimator worries most: well grounded, and the biggest mover on the page.
Three things worth naming directly, since this is where the real judgment sits. We considered putting the raw temperature number directly in front of authors, a slider from 0 to 1.2, and rejected it: most novelists have never heard of sampling, and a bare number either gets maxed out by curiosity or never touched at all. Instead, "continue the scene" ships with a labeled choice, Close to what I wrote, or Push it further, mapped internally to 0.75 and 0.9. The AI-specific failure worth guarding against is a kind of quiet confabulation: push temperature high enough and the model can invent a detail, a name, a place, a fact, that never existed in the author's own story bible. The guardrail is a same-request check that reads any new proper noun in a high-temperature output against the story bible before it's shown, and flags anything that isn't there. And there's a real cost to the higher settings, not just a quality upside: at 0.3, about 4 percent of outputs get silently rerolled for going off-track; at 0.8, that climbs to about 35 percent, roughly one extra model call for every three accepted paragraphs. Nobody gets a wider range of ideas and a cheaper feature at the same time.
And if you want to be sure it really works, try it somewhere else
Same five letters, a kitchen instead of a chapter. This time turning the dial up doesn't risk an ugly sentence, it risks a dish nobody can actually cook.
Larderloom's Substitute feature reads a recipe and a user's pantry, then suggests a swap for whatever's missing, before a cook gives up mid-recipe and drives to the store. Rozene Wintermere owns its settings, and runs the same five letters Vespera did.
Different product, same shape of question. What happens between "suggests" and "tries it" is where temperature does all its work.
Break it down: every substitution starts as a ranked list of candidate ingredients, and temperature decides how far down that list the suggestion is allowed to reach. Own the numbers: the same no-buttermilk prompt, run twenty times, matched the top suggestion, a mix of milk and lemon juice, 18 of 20 times at 0.2, and only 3 of 20 times at 0.9. Use a range: unlike scene continuation, Substitute doesn't get to chase maximum spread, because an ingredient swap outside the recipe's own cuisine costs someone dinner, so the recommended band tops out at 0.4 to 0.5, not 0.9. Nail the sanity check: home testers rated one in five suggestions generated at 0.9 as "I would never actually try this," against roughly one in twenty at 0.5, so the ceiling here is set by what a cook will actually attempt, not by how interesting the model can get. Direction: the swing assumption isn't the reroll rate or the sample size, it's how far outside the recipe's own cuisine a substitution is allowed to reach before it stops being a swap and starts being a different dish, and that's the number Rozene would fight hardest to get right.
The decision Rozene would take back
Substitute launched with the same 0.2-to-0.9 exploration range borrowed from an earlier, purely creative feature, one that suggests entirely new recipes from what's on hand. That range made sense for invention. It didn't make sense for a swap that has to actually work in someone's oven that night.
Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Say the shape fast: rerun the same prompt at a few settings, count how often it repeats itself, give each feature a range instead of one shared number, and check the range against a real reader before calling it done.
Cost: no budget for a forty-person blind read test this sprint. Ship the range anyway, labeled as unchecked against a real reader, and schedule the sanity check for the next release instead of skipping it.
The model got better, for real: say the underlying model gets meaningfully better at staying in-story even at high temperature. The range for "continue the scene" can push higher, maybe 0.8 to 1.0. It still stays a range. A better model earns a wider ceiling, not a single flat number.
Where people run it wrong.
They pick one company-wide temperature because four separate numbers feels like four separate arguments to have.
They treat a high setting as pure upside, creativity, without ever counting what it costs in silent rerolls.
They test the range against a spreadsheet number instead of a real reader, and never learn whether anyone can actually feel the difference.
How to use it live. Ask which job needs the same answer twice before naming any number. That one question tells you which half of the dial you're even arguing about.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework is this, and what's its one job?
Tap to flip
ANSWER
BOUND: turn a squishy word like predictable into a real mechanism, a counted number, a defensible range, and a check against what a person actually notices.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Vespera Fenwrick, who owns every temperature setting on Draftkiln's writing engine at Inkbramble Labs, and has to explain why "predictable" isn't one number.
3 · THE MECHANISM
What does turning temperature up actually change?
Tap to flip
ANSWER
It reshapes the odds behind every next word. Near zero, the model almost always repeats the single most likely word. Turned up, words that were only a little less likely get a real chance too.
4 · THE REAL TRADE-OFF
What's the actual trade-off temperature controls?
Tap to flip
ANSWER
Predictability against creativity: the same right answer twice, or a genuinely different idea each time. Draftkiln's bug was using one number to try to buy both at once.
5 · THE OLD DECISION
What decision would Vespera take back?
Tap to flip
ANSWER
Copying the continuity fixer's proven 0.3 setting into all four features at launch, because reusing a trusted number felt safer than picking four new ones.
6 · THE NUMBER
Fill in the blank: at 0.3, ___ of 20 reruns matched the same beat. At 0.8, only ___ did.
Tap to flip
ANSWER
18 of 20 at 0.3, 90 percent. 5 of 20 at 0.8, 25 percent. A single setting change moved the match rate 65 points.
7 · THE REPLAY
Same beta complaint, new ranges in place, what changes?
Tap to flip
ANSWER
Rerunning Enid's exact prompt at the new 0.8 setting lands only 5 of 20 on the same beat, and a blind read test finds readers call the 0.3-to-0.75 gap a genuinely different direction 83 percent of the time.
8 · CROSS-PRODUCT TRANSFER
Section 4 runs BOUND again on a different product. Which one, and what's the swing assumption there?
Tap to flip
ANSWER
Larderloom's Substitute, an ingredient-swap feature for home cooks. The swing assumption is how far outside a recipe's own cuisine a suggestion can reach before it stops being a substitution and starts being a different dish.
Check yourself Score: 0 / 0
True or false
1. True or false: since temperature is one setting, the same value should work fine across every feature built on the same underlying model.
True
False
Show hint
Look at the U step, and what happened when Draftkiln shipped one shared number.
Show answer
False. Draftkiln's four features needed four different ranges, from 0 to 0.15 up to 1.0 to 1.2, because they are not doing the same job.
Multiple choice
2. Why did Vespera score "same next beat" by having a person read the first two sentences, instead of checking for an exact text match across the 20 reruns?
A. Exact matching is slower to compute.
B. Two continuations can tell the same next beat in different words, so an exact-text match would undercount how often the model actually repeated itself.
C. Draftkiln's policy requires human review of all output.
D. The model's own confidence score already flags repeats.
Show hint
Look at the O step, "own the numbers."
Show answer
B. Two paragraphs describing the same event in different words would score as "different" under exact matching, which understates how often the model is really landing on the same idea, not just the same words.
Fill in the blank
3. At temperature 0.3, ___ of 20 reruns matched the same next beat (___ percent). At 0.8, only ___ did (___ percent).
Show hint
Check the O and N steps, and the stacked bar chart in Let's learn.
Show answer
18 of 20 (90 percent); 5 of 20 (25 percent). A single setting change, 0.3 to 0.8, moved the match rate 65 points, more than any other change tested.
Short answer, where it wouldn't matter
4. Name a place in Draftkiln's own four features where pushing temperature up would NOT be worth doing, and why.
Show hint
Look at "what I would leave alone" in Let's learn.
Show answer
Model answer: Fixing a continuity error or renaming a character consistently. Both need the exact same right answer every time you run them, so a different "fix" for the identical mistake on every rerun would be its own kind of broken. Those two stay near zero on purpose.
Short answer, apply it yourself
5. Think of an AI tool you've used that generates text, an email drafter, a chatbot, a caption tool. If it let you turn creativity up or down, what job inside that tool would you want kept near-deterministic, and what job would you want to actually vary?
Show hint
Ask which job needs the same right answer twice, and which one is only useful if it's different each time.
Show answer
Model answer: An email drafting tool: keep "fix my grammar" near-deterministic, since you want the same correct sentence back every time, but let "suggest three subject lines" run with real spread, since three near-identical subject lines are useless.
Short answer, work the number
6. If Draftkiln doubled its rerun sample from 20 to 40 at the same 0.3 setting, and the true match rate really is 90 percent, roughly how many of the 40 would you expect to match, and does that change the underlying decision?
Show hint
A bigger sample makes a measured rate more solid, it doesn't change what the rate means for the decision.
Show answer
About 36 of 40. It would not change the decision. A bigger sample makes the 90 percent more solid, it does not make 0.3 suddenly right for a feature whose whole job is variety.
Before you close the answer
Why this works
Tests whether you'll turn a soft ask like "predictable" into an actual mechanism and a number, instead of picking a comfortable-sounding setting and hoping. And whether you know a confident single number is a red flag when it's being asked to do two different jobs at once.
Follow-up traps
"Why not just expose temperature to authors and let them pick?" Response: most novelists have never heard of sampling temperature; a bare slider either gets maxed out by curiosity or never touched, so "continue the scene" ships a labeled choice instead, mapped internally to the real range.
"Isn't a wider range just a way of admitting you don't actually know the right number?" Response: no, it's the honest width a 20-rerun test and a 40-reader check actually support. A flat number would look more certain and say less.
If pressed
The rerun test isn't pure random sampling. At least a few of the twenty reruns are forced to start from a scene that already has three named characters in play, not just simple two-person scenes, because that's exactly where a high-temperature continuation is most likely to invent a detail the story bible doesn't have.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
