ConceptFoundationalModel Fluency & the AI PM Role / The AI literacy baseline every PM needs / #5
What does temperature control and when would you lower it in a product?
GUARD · a setting borrowed from a marketing template, tested on Corrivane Health's trial matcher, Sievelight
Sievelight is Corrivane Health's tool for matching cancer patients to open clinical trials. It reads a patient's chart, checks it against a trial's eligibility list, and writes the coordinator a plain-language reason the patient does or doesn't match, before she picks up the phone. Prendergast Oncology Alliance runs it across nine cancer centers. Vesalia Cathcart owns Sievelight's matching and explanation flow. She didn't expect a setting copied from a marketing-copy template to be the thing a glioblastoma patient's family stopped trusting.
The direct answer
Temperature controls how much the model gambles on its next word: low keeps it picking the safest, most likely word every time, so the same input reads almost the same way twice; high lets it take real chances on purpose, so the same input can come back worded differently, sometimes leaning on a different fact to make its point. Lower it, down near 0.2, for any output that has to read the same way twice, like a clinical-trial eligibility explanation a patient hears out loud. Leave it high only where the variety is the actual point, like a coordinator's opening line.
Do this, in order
Lower temperature to about 0.2 for the eligibility-explanation output specifically.Why: it's the one output that has to say the same thing twice for the same facts, not the one that benefits from sounding fresh.
Log each explanation with the chart snapshot, the trial criteria, the model version, and the temperature used.Why: without a stored record, nobody can tell whether wording changed because a lab result changed or because a dial did.
Keep temperature high wherever the output is genuinely creative.Why: a coordinator's opening line or phrasing-coaching tool should not read like it was typed once and photocopied forever.
Run a repeat-call diff test on the eligibility explanation before shipping any change.Why: it's the cheapest test there is, and the easiest one to never get around to running.
Reject caching one locked explanation instead of lowering temperature.Why: a cached explanation goes silently stale the moment the patient's chart legitimately changes.
Watch for material differences across repeated calls in production, not just once at launch.Why: a later model update can quietly bring the variance back after the first fix.
How to answer this, stage by stage
Nobody is grading whether you can define temperature like a textbook. They're grading whether you can say, plainly, which outputs in a real product need it turned down and why a patient would actually care.
1
Ground it in one real product, not the abstract setting
Say it like this
"Let's make this concrete. Sievelight is Corrivane Health's matching tool. It reads a cancer patient's chart, checks it against a trial's eligibility list, and writes a coordinator the reason a patient does or doesn't match, before she calls the family."
Why this works
A real product and a real call keep the answer from turning into a definition recited from a course.
2
Say your structure out loud
Say it like this
"I'll run this as GUARD. Groups, who's affected. Unequal, where the harm actually lands. Ability to contest, who can tell a real change from a random one. Reduce, the actual fix. Detect, how you'd catch it before a family does."
Why this works
Two seconds of structure signals you have a method, not just an opinion about a slider.
3
Answer the literal question, plainly, before anything else
Say it like this
"Temperature controls how much the model gambles on its next word. Every word it writes, it picked from a ranked list of options. Turn temperature down, and it keeps grabbing the top pick, so the same input gives you close to the same answer twice. Turn it up, and it takes riskier picks on purpose, so the same input can come back worded differently, sometimes leaning on a different fact entirely."
Why this works
The question asked what temperature controls. Answer that in one breath before the story starts, or you never actually answer it.
4
Name both people GUARD makes you name
Say it like this
"There are two people in this, not one. Vesalia, the applied PM, who set the dial. And a patient like Cosette, who only ever hears the words that dial produced, never the dial itself."
Why this works
Naming the person with no lever is the strongest move GUARD has. Skip it and the answer stays a technical footnote.
5
Reframe what the question is really testing
Say it like this
"This sounds like 'what's temperature.' It's really 'which outputs need to sound the same twice, and which ones are allowed to sound different.' A friendly opening line can vary all it wants. The actual reason a patient is excluded from a trial cannot."
Why this works
This line is the whole answer in miniature. The short version is above; the long version, the story of what happened at Prendergast, is what proves it below.
6
Give the committed answer
Say it like this
"So here's what I'd do. Split the setting. Turn temperature down, around 0.2, for the eligibility explanation itself, so it's reproducible and something you could put in front of an auditor. Leave it high wherever the output is genuinely creative, like coaching language for how to phrase hard news."
Why this works
This is deliverable 0, spoken, before a single number distracts from it.
7
Prove it with the real numbers, not the abstract case
Say it like this
"Here's what happened at Prendergast. Sievelight ran the eligibility explanation at 0.9, the same setting used for patient-facing marketing copy. We ran the same patient's record through it ten times back to back: all ten came back worded differently, and four of the ten led with a different exclusion reason than the others, for a chart that never changed. At 0.2, nine of ten came back nearly word for word, and none of them changed the lead reason."
Why this works
A real repeat-call test, with real numbers, beats any amount of talk about randomness in the abstract.
8
Close on something checkable, not just a definition
Say it like this
"So: lower it for anything that has to be reproducible, leave it up for anything that's genuinely creative. You'll know it's actually fixed when the same record run through Sievelight ten times comes back saying the same thing ten times, not when someone tells you it feels more consistent."
Why this works
Ends on a test anyone could run, not a feeling.
Let's learn
What happens when the exact same clinical fact gets written up two different ways, on two different calls, and nobody told the patient that could happen?
Sievelight reads a cancer patient's chart, works out which of a trial's inclusion and exclusion lines the patient clears and which they don't, and writes a coordinator a short, plain-language reason before the call. It answers in about 40 seconds. That timing doesn't change with the setting in question here; only the wording does.
Same input, twice. One setting answers it the same way both times. The other doesn't, on purpose.
Knowledge spark: what is temperature, really?
Every time a language model writes a word, it's really choosing from a ranked list of likely next words. Temperature decides how often it takes something other than the top pick. Near zero, it almost always takes the safest word. Near one, it takes real chances on purpose, which is what makes writing sound less repetitive and more human, right up until the moment you needed it to say the same thing twice.
Before Sievelight, a coordinator checked a patient's chart against a trial's criteria by hand, about 25 minutes per patient per trial, reading lab values against a list that can run twenty or more lines long. Sievelight cut that to about 6 minutes of reading and light editing. That part of the story is a genuine win, and it hasn't changed.
What changed is what a patient hears on two different calls about the exact same chart. Corrivane had set Sievelight's eligibility-explanation generator to 0.9, the same temperature used across its patient-facing copy line, marketing emails, portal messages, the kind of writing that's supposed to sound a little different every time. Nobody had split it out for the one output that isn't supposed to.
Share of repeated outputs that differ from the first response, by temperature setting, same patient record
Share of repeated outputs that differ from the first
Sievelight shipped at 0.9, near the top of this curve, chosen for the marketing copy it was borrowed from, never re-checked against an output that had to hold still.
We didn't give a patient a wrong answer. We gave her two right-sounding ones that disagreed with each other.
What that costs at its worst: a family hearing two different-sounding reasons for the same rejection, days apart, doesn't conclude "the wording changed." They conclude something about the patient's own situation changed, or that one of the two calls was a mistake. Either way, they stop trusting the next thing they're told, at the exact moment a fast-moving diagnosis needs them moving, not double-checking.
The choice I would take back
When Sievelight's explanation generator was first built, it inherited its temperature from Corrivane's shared patient-facing copy template, the same 0.9 used for warm, varied marketing writing. That felt harmless, since nothing about an eligibility explanation seemed different from any other patient-facing sentence. It stayed unquestioned for exactly as long as nobody compared two calls about the same chart side by side.
What I would leave alone: the line Sievelight drafts to open a call, "Hi, thanks for taking a minute today." That one can say something slightly different every time and nothing is lost, no fact is riding on it, no chart line depends on it reading the same way twice. Same for the separate tool that suggests a few ways to phrase hard news to a coordinator in training. Variety is the actual point there. This fix doesn't touch either one.
The lesson: a setting that's invisible in a demo can be load-bearing in production. Nobody sees "temperature: 0.9" and pictures a patient hearing two different reasons for the same rejection. You only see it once you run the same input twice and actually compare the two answers, which almost nobody does by default.
Now here is the same thing as a story
The short version above is what you actually say in the room. Read this one for the week it cost before anyone compared two calls side by side.
Thomasina Kinnear has run intake calls at Prendergast's flagship center for five years. She knows most of the Alliance's open trials by heart, the biomarkers, the age cutoffs, the labs that matter. She's good at the hard part of the job too, telling a family "not a match" in a way that still sounds like someone is on their side.
Sievelight arrived in her queue eighteen months ago. For most of that time it was the best change to her week. She'd open a chart, glance at Sievelight's draft, read it once against the chart to be sure, then call. Twenty patients some mornings, done by early afternoon instead of running calls into the evening.
Nobody decided to stop checking. It thinned out in three ordinary weeks, the way habits do.
She read every draft twice against the chart, for months. Sievelight never got a fact wrong that she caught, so the second read started to feel like a formality. She dropped to one read. Some weeks, she read it straight into the phone, trusting it the way you trust a colleague who's never once let you down.
Then came Cosette Fontrell, 43, three weeks past the biopsy that confirmed a glioblastoma, with a phase 2 trial that closes enrollment five weeks after that date. Her platelet count sat under the trial's cutoff by a small margin, and her neutrophil count was borderline too, close enough that either one could plausibly be the sentence Sievelight chose to lead with.
Vesalia set the dial. Thomasina read what it produced, out loud, in good faith. Cosette only ever heard the words.
On the Thursday call, Sievelight's draft led with the platelet count. Thomasina read it as written: "You're not currently eligible, mainly because your platelet count came in under what this trial allows." Cosette's sister, who works with data for a living, wrote it down.
Nothing changed in Cosette's chart that week. No new labs, no new note. But a follow-up call went out the next Thursday to confirm the family understood the next steps, and Sievelight redrafted from scratch, the way it does on every call. This time it led with the neutrophil count instead: "Mainly because of where your white blood cell counts are sitting right now." Same chart. Different sentence, different reason foregrounded, both technically true, presented as if one had replaced the other.
Cosette's sister caught it in seconds. "Last week you said it was platelets. Now it's white cells. Which is it? Did something change?" Thomasina didn't have an answer, because nothing had. She pulled both call notes and the two didn't line up, not because either one was wrong, but because Sievelight had picked a different lead each time from a chart that hadn't moved.
Five steps between an unchanged chart and a family hearing something that sounds new. The fourth one, where anyone could catch the mismatch, was never built.
What it actually cost: nearly a week. Cosette's family spent that week re-reading both explanations, calling the clinic twice more to ask which was true, and holding off on a second trial site they'd been ready to call, because they no longer trusted that Prendergast's answer meant what it said. By the time Thomasina sorted it out and confirmed the real, single reason, Cosette's five-week enrollment window had eight days left in it instead of the two weeks the family thought they had.
We didn't take a fact from Cosette's family. We took the week they would have spent finding a second trial instead of arguing with the first answer.
Vesalia remembers the meeting where 0.9 got chosen, sort of, because it wasn't really a meeting about this at all. It was a meeting about Sievelight's patient-portal messages, the ones that remind someone about an upcoming appointment or thank them for a call. Someone on the copy side wanted those to sound less like a form letter, so they raised the temperature until the wording varied naturally. The eligibility-explanation generator was built afterward, on the same underlying prompt template, and nobody split its setting out, because nobody was in the room thinking about a fact that had to hold still.
The decision Vesalia would take back sits there, months before Cosette ever called. Not the model. Not Thomasina's habit of reading it once instead of twice. The shared 0.9, applied to an output that was never supposed to be allowed to vary.
Run the same two Thursdays again, with the explanation generator at 0.2 and a logged snapshot of what the chart looked like at the moment each explanation got written. Sievelight still writes a fresh explanation on both calls. But now the wording holds close to identical, because there was nothing new to gamble on, and if the chart genuinely had changed between calls, the log would show exactly what changed and why the wording moved. Same product, same speed, a reason that reads the same way twice because the facts hadn't moved.
The plain difference: one design let a coin flip inside the model decide which true fact a family heard first. The other one only lets the chart decide that.
What I'd tell myself, back in that copy meeting: a setting that makes a reminder email sound warmer is free. The same setting, left untouched on an output someone's treatment decision depends on, is not free at all. We just hadn't found the bill yet.
GUARD, for a dial nobody told the patient existed
This isn't really about Thomasina's habit of reading a draft once instead of twice. GUARD is for naming who's exposed when a setting like this is wrong, and what specifically to change about it.
GGroups. Who is affected, and how.
Three people, not one. Vesalia Cathcart, the applied PM who set the temperature and can change it. Thomasina Kinnear, the coordinator who reads Sievelight's words out loud in good faith, with no way to know they were gambled on. And Cosette Fontrell, the patient, who only ever hears the sentence, never the dial or the model version behind it.
Name the one who holds the setting and the one who only hears its output. Most answers only name one of the two.
UUnequal. Where the harm concentrates, and on whom.
It doesn't land evenly across every patient Sievelight explains a match for. It concentrates on patients like Cosette, whose charts are borderline on more than one criterion at once, because that's exactly the situation where the model has more than one true fact it could gamble on leading with. A patient who clearly matches or clearly doesn't never sees the variance, because there's only one honest sentence to write either way.
This is what makes the harm structural, not bad luck. The patients with the least clear-cut cases are the ones most likely to get two disagreeing explanations.
The fix isn't one dial for the whole product. It's knowing which quadrant each output actually sits in.
AAbility to contest. Who has a lever, and who has empty hands.
Vesalia has the dashboard and the setting. Thomasina has a phone and two call notes she can compare, once she thinks to. Cosette's family had neither. They had no way to know that a changed sentence about their own family member's medical eligibility could come from a sampling setting instead of a real change in her chart, because nothing they were shown ever said the explanation might reword itself.
This is GUARD's sharpest move: naming who can't tell a real change from a random one, before it costs them a week.
RReduce. The actual fix, not a policy memo.
Split the setting by what the output is for. Lower temperature to about 0.2 for the eligibility-explanation generator specifically, so the same chart produces close to the same sentence every time. Leave it at 0.9 for outputs where variety is the entire point, the opening line, the phrasing-coaching tool coordinators use in training. And log every explanation with the chart snapshot and model version behind it, so a reworded sentence can always be traced to a real change or ruled out as one.
The alternative worth naming and rejecting: cache the very first explanation ever written for a patient and always replay that exact text. It sounds simpler, but it goes silently stale the day the patient's real labs change, which is the one day the explanation actually needs to update. Lowering temperature keeps the explanation fresh from the current chart while making it stop gambling on how to word it.
DDetect. How you'd know, before a family does.
Run the same patient record and trial through Sievelight ten times back to back. At 0.9, all ten came back worded differently, and four of the ten led with a different exclusion reason than the others, for a chart that hadn't moved. At 0.2, nine of ten came back nearly identical, and none changed the lead reason.
This test costs nothing and catches everything the Cosette call didn't. The failure worth naming plainly: non-deterministic sampling can hand a family a different-sounding clinical explanation for an input that never changed, and it will read as new information unless someone specifically tests for it.
Repeated calls that named a different lead reason than the first call, same record, ten calls each setting
Same record, same trial, ten repeated calls each way. The only thing different between the two bars is the dial, not one fact in Cosette's chart.
The test that keeps this honest
If the fix here were "add a disclaimer that wording may vary," nothing would actually change, that's a sentence, not a design decision. The trade-off worth naming out loud: lowering temperature doesn't cost latency or extra compute, the model still writes the same number of tokens either way. What it costs is variety, the explanation text reads a little more repetitive across many different patients' calls than it did before. That's a real trade, accepted on purpose, for the one output where sounding the same twice is the actual requirement.
And if you want to be sure it really works, try it somewhere else
Same five letters, a small-business bank instead of a cancer center, and the thing that changes call to call this time is why a loan got declined.
Ledgerscope, built in-house at Oakmarsh Community Bank, reads a small-business loan application and drafts the reason it was declined or flagged for more documents, for a loan officer to read on the callback. Dovina Estregard owns that flow the way Vesalia owns Sievelight's.
Different desk, same missing split between an output that has to hold still and one that never needed to.
Ledgerscope shipped with a single 0.85 temperature across every output it wrote, borrowed from the same team's automated review-response generator, where a little variety reads as more personal. When a business owner's application sat close to two cutoffs at once, debt-to-income and time-in-business, a follow-up call sometimes cited a different one of the two than the first call had, for an application that hadn't been resubmitted or changed. A federal adverse-action notice requires the actual, specific reason a credit decision was made, and two different specific reasons for one decision is exactly the kind of inconsistency a regulator asks about.
Same rank, mapped onto Ledgerscope: lower temperature to about 0.2 for the decline-reason generator specifically, log the application snapshot and model version behind every notice, and leave the higher setting only on outputs like the loan officer's own internal call-prep notes, where a little variation costs nobody anything.
Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: lower temperature for anything that must be reproducible and auditable, keep it high only where variety is the actual product.
Cost: no budget this quarter to touch the pipeline. Change the one config value on the decline-reason call this sprint; it's a settings change, not a rebuild, and it costs nothing in latency or compute either way.
The model got better, for real: say the newest version is measurably more accurate across every criterion. Keep the low temperature on the decline reason anyway. A more accurate model can still gamble on which of two true reasons to lead with, and accuracy doesn't fix that.
Where people run it wrong.
They treat temperature as a single, product-wide setting instead of asking which specific outputs need to be reproducible.
They add a disclaimer about wording instead of actually fixing the setting that's producing the disagreement.
They watch for the model being factually wrong and miss that it can be factually right, twice, in two different-sounding ways that read as a contradiction.
How to use it live. Before answering, ask yourself out loud: "does this output need to say the same thing twice for the same facts, or is the variety itself doing work?" Say which, for the specific output in the question, and the right temperature answer usually falls right out of it.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
Which framework fits "what does temperature control and when would you lower it"?
Tap to flip
ANSWER
GUARD: groups, unequal, ability to contest, reduce, detect. It fits because the real question isn't the definition, it's who gets hurt when the setting is wrong for the use case.
2 · THE PEOPLE
Who are the three people this answer names?
Tap to flip
ANSWER
Vesalia Cathcart, the applied PM who sets Sievelight's temperature. Thomasina Kinnear, the coordinator who reads its output out loud. Cosette Fontrell, the patient who only ever hears the words.
3 · THE HABIT
What did Thomasina stop doing because Sievelight never got a fact wrong that she caught?
Tap to flip
ANSWER
Reading each draft twice against the chart before calling. It thinned to one read, then some weeks to reading it straight into the phone.
4 · THE SETTING
What's the two-setting switch in this story?
Tap to flip
ANSWER
Temperature near 0 keeps Sievelight's wording close to fixed for the same chart. Temperature near 1 lets it pick a materially different phrasing, and sometimes a different lead reason, for the exact same record.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Sievelight's eligibility-explanation generator inherited the 0.9 temperature from Corrivane's patient-facing marketing copy template, and nobody split it out for the one output that had to read the same way twice.
6 · THE NUMBER
Fill in the blank: at temperature 0.9, ten repeated calls on the same record produced ___ of 10 outputs worded differently, and ___ of 10 that led with a different reason.
Tap to flip
ANSWER
10 of 10 worded differently. 4 of 10 led with a different reason. At 0.2, that dropped to 1 of 10 (punctuation only) and 0 of 10.
7 · THE REPLAY
Same two Thursdays, new setting, what changes?
Tap to flip
ANSWER
At 0.2 with a logged chart snapshot, both calls read close to identical, because there's nothing new to gamble on. The family never loses trust, and Cosette keeps her full enrollment window instead of eight days of it.
8 · CROSS-PRODUCT TRANSFER
Section 4 runs GUARD again on a different product. Which one, and who plays the equivalent roles?
Tap to flip
ANSWER
Ledgerscope, Oakmarsh Community Bank's loan-decline explainer. Dovina Estregard plays Vesalia's role, owning the setting that let a decline reason shift call to call.
Check yourself Score: 0 / 0
Fill in the blank
1. At temperature 0.9, running Cosette's unchanged record through Sievelight ten times produced ___ of 10 outputs worded differently, and ___ of 10 that led with a different exclusion reason.
Show hint
Check stage 7 of the walkthrough, and the bar chart under the Detect step.
Show answer
10 of 10, and 4 of 10. At temperature 0.2, that dropped to roughly 1 of 10 differing (punctuation only) and 0 of 10 changing the lead reason.
Multiple choice
2. Why does lowering temperature for the eligibility explanation not cost extra latency or compute?
A. Corrivane upgraded to faster servers at the same time.
B. Temperature changes how a word gets picked from the model's ranked options, not how many words get generated.
C. A lower temperature always finishes generating faster.
D. Sievelight caches the answer instead of generating it fresh.
Show hint
Check the knowledge spark on what temperature is, and the trade-off named in the GUARD recap's key point.
Show answer
B. Temperature is a sampling choice, not a change to how many tokens the model writes, so it's essentially free to change. The real trade-off is variety of wording, not speed or cost.
True or false
3. True or false: temperature should be turned down everywhere in Sievelight, including the line that opens each call, since consistency is always safer.
True
False
Show hint
Check "what I would leave alone" in Let's learn, and the quadrant diagram in the GUARD recap.
Show answer
False. The opening line and the phrasing-coaching tool carry no fact that has to hold still, so a high temperature there costs nothing and adds warmth. Only outputs that must be reproducible need the lower setting.
Short answer, name the rejected alternative
4. Besides lowering temperature, what alternative fix does this answer name and reject, and why does it lose?
Show hint
Look at the Reduce step in the GUARD recap.
Show answer
Model answer: Caching the first explanation ever written for a patient and always replaying that exact text. It loses because it goes silently stale the moment the patient's real chart changes, which is exactly the moment the explanation needs to update.
Short answer, apply it yourself
5. Think of an AI product you use or have built that generates the same kind of explanation more than once for the same underlying facts, a denial reason, a recommendation, a status update. Should that output be reproducible, and if you ran it twice right now, do you actually know whether it would come back the same?
Show hint
Look for any place a model explains a decision in words, more than once, about the same underlying facts.
Show answer
Model answer: A customer-support bot that explains why a refund was denied. If a customer calls back and gets a different-sounding reason for the same order, that's the identical failure, just retail instead of clinical, and most teams have never actually run the repeat-call test to check.
Short answer, work the number
6. If Sievelight's explanation takes about 40 seconds to generate at any temperature setting, how many extra seconds does lowering it from 0.9 to 0.2 cost per call, and what does that tell you about the real trade-off being made?
Show hint
Reread the note in Let's learn about generation time not changing with the setting, and the key point in the GUARD recap.
Show answer
Zero extra seconds. The trade-off isn't speed or cost at all, it's that wording reads more repetitive across many different patients' calls. That's a real cost, but it's a tone cost, not a latency one, and confusing the two leads people to assume the fix is expensive when it isn't.
Before you close the answer
Why this works
Tests whether you actually understand temperature as a mechanism, not just as a knob to "turn down for safety," and whether you can tell which specific outputs in a real product need that. Most candidates can gesture at "randomness is risky." Naming exactly which output needs reproducibility, which doesn't, and what it costs to fix, is the part almost nobody does unprompted.
Follow-up traps
"Why not just turn temperature to zero everywhere, to be safe?" Response: because outputs like the opening line or the phrasing-coaching tool exist specifically for their variety, and a temperature of zero there makes every call sound scripted for no safety benefit at all.
"Isn't this really just a testing gap, not a temperature problem?" Response: both are true and neither replaces the other. The repeat-call test is how you'd have caught it, but the fix that stops it recurring is the setting itself, since a test alone doesn't change what ships next time.
If pressed
Even at temperature near zero, some model serving stacks aren't perfectly deterministic, batching and floating-point rounding on the backend can still nudge an output slightly. Corrivane's actual fix pinned the model version and a fixed random seed per explanation alongside the lower temperature, not the temperature alone, to get true call-to-call reproducibility rather than just a much lower chance of drift.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.
