What is the risk of an onboarding that oversells capability?

The report is one page. It says HullSense Verified Diagnosis, and nothing about what it didn't check. Imelda Reyes manages service at a boatyard that runs HullSense on every engine that comes in for a checkup.

How to answer this, stage by stage

Six stages. Say each one plainly, without moralizing, and the answer holds up under a real follow-up.

Let's learn

The report is one page. It says HullSense Verified Diagnosis, and nothing about what it didn't check.

HullSense is a diagnostic tool a boatyard runs on an engine during a checkup, reading sensor data and comparing it against known failure patterns to name what's wrong before a mechanic tears anything apart.

On the boatyard's most common engines, four outboard motor lines that make up most of the fleet, HullSense gets it right about 94 times in 100. On older diesel inboards, a smaller and less common slice of boats, it's right about 68 times in 100, because far fewer of those engines exist in the data it learned from.

At its worst: a boat owner with an older diesel inboard nearly authorized a full engine teardown, a repair costing thousands, based on a confident HullSense call. A newer mechanic on shift that day happened to double-check the sensor readings by hand before the work order went out, and found a faulty temperature sensor, not the engine damage HullSense had named. Nothing on the report had told the owner, or the mechanic, that this engine type was one HullSense handled worse.

What I would leave alone: for the four common outboard lines, where HullSense really is right 94 percent of the time, a strong, confident badge is honest and it should stay. The fix is naming where that confidence doesn't hold, not softening it everywhere.

The lesson: "verified" is a claim about the whole system. If any part of that system is weaker for some customers than others, the badge is making a promise the model itself never made.

Now here is the same thing as a story

The short version above is the plain risk case. Read this one for how close the near miss actually came.

The clipboard behind Imelda's desk has a HullSense printout stapled to every third page, one for nearly every engine that's come through this year. She's run service at this boatyard for twelve years, long enough to know most of these engines by the sound they make idling.

Imelda can read the sensor data behind a HullSense call. The owner across the counter only ever sees the one word: Verified.

HullSense launched with a clean, confident report design, on purpose. New customers were nervous about trusting an algorithm with their engine, and a flat "Verified Diagnosis" badge, without any hedging, was what got people comfortable fast. It worked. Business grew.

Same underlying model, same sensor data. The only difference is whether the report admits where its own confidence actually sits.

Nobody at the boatyard was hiding the accuracy gap between common outboards and older diesels. Nobody had measured it as a gap at all, until the near miss.

The bottom right corner is where the real danger lives: high stakes, paired with a real accuracy rate nobody had put a number on yet.

A boat owner with a 1994 diesel inboard brought his engine in running rough. HullSense's report named a cracked engine block, a repair running into the thousands, with the same confident badge every other report carried. The owner, no engineer, had no reason to question it. He was one signature away from authorizing the teardown.

A newer mechanic on shift, still in the habit of double-checking sensor readings by hand before big jobs, happened to catch it: a faulty temperature sensor was throwing a reading that looked exactly like block damage to the model. The real fix cost around 200 dollars, not several thousand.

The gap wasn't in step one, two, or three. It was the missing fourth step, the one nobody had ever built.

We did not almost lose an engine. We almost sent a customer thousands of dollars into a repair he had no way to question, because nothing on the page ever told him he was allowed to.

Four additions to one page. None of them touch the model itself, only what the page is honest about.

The near miss is the milestone that mattered. Everything after it was a direct response to that one afternoon.

I built the confident badge because early customers needed reassurance, and it worked exactly as designed. It took one mechanic's habit of double-checking, a habit the product itself did nothing to encourage, to catch what the badge was quietly hiding from everyone else.

GUARD, applied plainlyNot a policy. GUARD is what forces the design to say who can push back, and who can't.

The recap, one line per letter: groups is the mechanic and the owner, unequal is the 26-point gap between engine types, ability to contest is the missing second-look path, reduce is printing a range and a contact on the report itself, and detect is watching the request rate for a signal that's suspiciously calm.

And if you want to be sure it really works, try it somewhere elseSame five letters, a rental-property inspection tool instead of an engine. This time the subject is a tenant, not a boat owner.

Callum Osei manages a portfolio of rental units and uses an AI condition-assessment tool to inspect a unit at move-out and decide how much of a deposit gets withheld for damage.

Mapped onto GUARD: groups is Callum, who reviews the tool's report, and the tenant, who has already moved out and has no way to be present for a re-inspection. Unequal is that tenants in older units, where wear patterns look more like damage to a model trained mostly on newer units, lose more of their deposit on average. Ability to contest is the real gap: many tenants don't know a dispute process exists at all, since the deposit letter states a dollar figure with no mention of how it was reached. Reduce is printing the specific findings with photos and a plain dispute deadline on the letter itself, not buried in a lease clause. Detect is tracking dispute rates by unit age, watching for the same silence HullSense had.

Different building, same missing lever: a tenant who never learns they could have asked for another look.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "name the confidence range, give a way to contest it," and stop.
Cost: there's no budget to redesign the report this quarter. Say so honestly, and start with one added sentence naming the engine types HullSense is weakest on, since even one honest line beats none.
The model gets better, for real: if HullSense's overall accuracy climbs to 97 percent, the ability-to-contest gap still matters, a rarer wrong call on a major repair is still a major repair nobody could question.

Where people run it wrong.
They treat this as a model-accuracy problem and try to fix it by retraining, when the actual gap is in what the report tells the reader.
They assume a low complaint rate means the system is trusted, when it often means nobody knew a complaint was possible.
They respond to a near miss with a review board instead of a specific, visible change to the document itself.

How to use it live. When someone asks about the risk of overselling capability, ask yourself one question first: who reads this output with no way to check it against anything else. Name that person before you name the model's accuracy number.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0