When does showing the model's reasoning help, and when does it reduce trust?

Trestlework matches a homeowner with a contractor for a repair job and shows a line explaining the match. Emrys Talbot booked emergency plumbing through the app at 1am after a burst pipe, standing in his kitchen in sock feet, watching water spread across the floor.

How to answer this, stage by stage

Nobody's grading whether you know AI reasoning can be shown. They're grading whether you can name the exact line it crosses from helpful to alarming.

Let's learn

Trestlework's matching assistant looks at a homeowner's job, checks nearby contractors, and shows the top match with a line explaining why.

Before the reasoning line existed, homeowners saw only a star rating and a "recommended" tag, and typically called two or three other services themselves to compare, about twenty minutes for an emergency job. With a short, specific reasoning line, "Licensed for gas lines, 12 minutes away, available now," that dropped to under two minutes, and booking completion for emergency jobs rose sharply.

Later, engineers added a "See full reasoning" expandable, showing the model's actual internal deliberation text verbatim for anyone curious. The turn: the raw text wasn't wrong, and it wasn't dishonest. The real problem is what a person does the moment reasoning stops sounding like a fact and starts sounding like a guess.

At its worst: a homeowner with a burst pipe reads a paragraph like "Contractor B is slightly cheaper, but I'm not fully confident in the availability window given limited recent data," panics at the word "not confident," and calls an unvetted number found through a quick search instead, at 1am, with no background check and no fixed price.

What I would leave alone: for a routine, planned-ahead job, like booking a gutter cleaning next month, the same raw reasoning panel doesn't hurt completion at all. Without urgency, hedging language reads as careful, not alarming.

The lesson: reasoning that's honest about its own uncertainty is not automatically reasoning that helps a person decide. Those are two different jobs, and a single panel can't always do both.

Now here is the same thing as a story

The short version above is what you'd say defending the fix to Trestlework's product team. Read this one for how the pattern actually showed up.

Emrys Talbot had used Trestlework for small jobs, a leaky faucet, a stuck garbage disposal, always trusting the top match without a second thought. The short reasoning line was enough. He'd stopped comparing contractors himself months earlier; the app was simply faster and always seemed right.

Same contractor, same match, two very different panels underneath it.

There was no single bad match that started this. Over about ten weeks after the "see full reasoning" panel rolled out broadly, Trestlework's product data quietly showed something odd: the emergency-job category specifically was completing bookings less often, even though the matches themselves hadn't gotten any worse.

Emrys's own night was one data point inside that climbing line. His pipe burst just after midnight. He tapped "see full reasoning" out of pure curiosity, something he'd never done before, and read a paragraph that hedged between two contractors, ending with a line admitting the model wasn't fully confident in one contractor's current availability.

Nothing in this chain was false. Each step was still enough to lose him.

He closed the app. He searched "emergency plumber near me" instead, called the first number that answered, and paid nearly double for a same-night visit with no vetting behind it at all.

Here's the decision I'd take back. We built "full transparency" into the product's early pitch, and treated every version of "show more of the model's own reasoning" as a straightforward improvement on that promise. Nobody separated "honest" from "reassuring," because for months, on non-urgent jobs, they'd looked like the same thing.

One panel is a window onto real, checkable facts. The other is a window onto the model's own second thoughts.

I'd split the panel in two. The default stays the short, fact-based line, for every job, every time. The raw reasoning moves behind a second, clearly labeled tap, "See the technical reasoning," so a curious user can still find it without it landing in front of someone mid-emergency.

The same underlying reasoning, routed four different ways depending on what's actually at stake in the moment.

Replay the same burst pipe under the new design: Emrys sees "Licensed for gas lines, 12 minutes away, available now" and books in under a minute. The full reasoning is still there, one tap further in, for the night he's calm enough to want it.

We chased "more transparency" because it was easy to defend in a room. It took one climbing chart, and one paragraph that talked a real person out the door at 1am, to see that transparency without a sense of timing isn't honesty. It's just noise at the worst possible moment.

LEAD, the number that moves firstNot a vibe check on "too much detail." LEAD is what tells you exactly which reasoning number to watch, and when to act on it.

Same two panels, four very different corners, and only one of them is genuinely dangerous.

The recap, one line per letter: link is a completed, vetted booking, early signal is the climbing abandon-after-reasoning rate, abuse is contractors writing toward the trigger words, and decision is the 15% threshold that flips a category's default back to the short blurb.

And if you want to be sure it really works, try it somewhere elseSame four letters, a university financial-aid office instead of a repair marketplace. A very different building, and the leading signal is a phone call, not a tap.

A university's financial-aid office uses an assistant that drafts a reasoning summary for why a student's aid package changed year to year, before a human advisor reviews and sends it. Mapped onto LEAD: link is whether a student actually reads and acts on the explanation, appealing correctly or accepting it, rather than calling the office in a panic. Early signal is the share of students who read the summary and then call the office within the hour asking "so is my aid being cut or not," which rises well before formal appeal volume does. Abuse: some advisors, once they see which phrases in the summary trigger fewer panicked calls, start pasting the same reassuring boilerplate onto every case regardless of whether it actually fits that student's situation. Decision: once the confused-call rate for a given change type crosses a set threshold, that change type's summary template gets rewritten in plainer, less hedged language, not just resent as-is.

A repair marketplace and a financial-aid office look nothing alike. The same three warning signs still apply to both.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "short and factual by default, raw reasoning gated behind a tap, watch the abandon-after-reasoning rate," and stop.
Cost: there's no budget to build a separate gated panel this quarter. Start by simply rewriting the existing raw text to remove hedging language like "not fully confident," even without changing where it sits on the screen.
The model gets better, for real: if the underlying match quality improves, the hedging problem doesn't go away on its own. A better model can still write an honest sentence that reads as doubt to someone under pressure.

Where people run it wrong.
They treat all reasoning as equally helpful, regardless of how urgent the moment is.
They measure whether people opened the reasoning panel, never whether they abandoned right after.
They assume more raw detail is always more honest, when a shorter, fact-based line can be just as true and far more usable.

How to use it live. When someone asks when reasoning helps versus hurts, don't answer with a general rule about detail. Ask what the person is doing in that exact moment, and whether hedging language would read as careful or as alarming to them right then.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0