What should you tell users when model behaviour changes underneath them?

How to answer this, stage by stage

Seven moves, from naming one real product to the line you would close on.

Let's learn

Say a dev-tools company builds a plugin that reads the code you are typing and writes the next few lines for you.

A year in, a regular user takes about seven of every ten suggestions the plugin gives, sometimes with a small tweak. Writing a validation check by hand takes about ninety seconds. Taking the suggestion takes about ten.

Three months ago, the company swapped the model that writes those suggestions for a smaller, faster one. Nobody told existing users. For someone who joined last month, nothing about their day changed enough to notice. For someone who has typed the same kind of function every morning for a year, the acceptance rate did not dip a little. It fell by half.

This is a trade someone chose on purpose. A smaller model answers faster and costs less to run, for everyone. It also does not write validation code the same way the old one did, and the person who feels that first is the one who trusted the old shape the most.

At its worst, that costs real time and real trust. Two of the changed suggestions got accepted and shipped, because they looked close enough at a glance and nobody reads that closely a year in. One of them quietly defaulted a missing stop address to nothing instead of stopping the code with an error. A truck sat idle at a transfer station for half a day while a dispatcher tracked the real address down by hand, and four other stops on that route slid to the afternoon.

What I would leave alone. A suggestion for a variable name, or the next line of an import list, carries no real judgment call either way. There is no old shape a user has memorized for typing "import" and waiting for the next library name. Swapping the model underneath that kind of suggestion is invisible, and it can stay that way.

The lesson. A better overall score is a fact about the model. It is not a fact about whether the one pattern a specific person built a year of habit around is still there. If the swap is invisible to the team that shipped it, do not assume it is invisible to the person who typed the same kind of function every day for a year.

Now here is the same thing as a story

The short version sits above. Read this one for the morning Deshawn's truck sat empty at a transfer station.

Deshawn can spot a missing check before he finishes reading a diff. Six years writing backend code for a trucking dispatch system, mostly the part that turns a morning route sheet into stops a driver's tablet can read. When something breaks at two in the morning, he is the one people call.

Pathlight showed up on his cracked-hinge laptop about a year ago. For months it was the best part of his morning. He would sit down at 8:10, type a comment like # validate stop before dispatch, hit tab, and watch a defensive check appear, the kind that stops the code cold the moment a required field is missing. He would read the first two or three completions of the day closely, just to see it was still behaving. It always was.

So he stopped reading the first two or three. Then he stopped reading most of them at all. By month eight he typed the comment, hit tab, and moved on, the same way he trusted a spell-checker to catch a typo.

There was no single morning where it changed. Anvilcode swapped the model under Pathlight sometime that spring, and nothing about the plugin looked different. Same button. Same screen. Only the shape of what came back was new.

One side can flip the lever. The other only finds out what it did.

Two completions that month wrote the check a different way. Instead of stopping when a stop was missing its drop-off address, the new shape filled it in with nothing and kept going. Deshawn accepted both, the way he had accepted a thousand completions before them, because they looked close enough at a glance and he had stopped reading closely months back.

One of those shipped. A truck went out with a blank stop on its route. It sat at the transfer station for half a day while dispatch called around trying to find the real address, and four other stops on that route got pushed to the afternoon.

Deshawn did not think Pathlight had a bug. He did not think he had made a mistake either, not at first. He genuinely could not tell which one it was, because nothing in the product had ever told him the model underneath had changed at all.

The step that should have caught him, and never got built

Weeks earlier, when Tessa Freeling's team debated the rollout, someone floated pinning the old model in place forever for any account with real history on it, so nobody who already trusted Pathlight would ever feel a swap. Tessa turned it down. Running two full models in production meant testing two models and shipping every future fix twice, for as long as either account existed. It felt like the safer choice in the room, and it was the wrong one to build a company around.

What she did not build instead, not yet, was any way to tell a user their own completions had changed shape. That is the gap I would go back and close.

Play the same spring forward with that gap closed. The model still swaps. But the first time Deshawn's validation completions come back looking different, a small line appears right in his editor: "Completions for missing-field checks changed with this update. They now fill in a default instead of stopping with an error. Review the first few, or turn on match my earlier style below." He reads it once, flips the toggle, and goes back to trusting the shape he built a year on. No blank stop ships. No truck sits for half a day. Dispatch never even hears about it.

What I would tell myself, back in that rollout meeting: a swap that never crashes and never errors can still cost someone a working morning. The interface did not break. It just quietly stopped matching what one specific person had learned to expect from it, and that kind of change needs its own announcement.

GUARD, for a swap nobody announced

This reads like a rollout-timing question. The real test is who meets the change first with no way to tell what happened.

And if you want to be sure it really works, try it somewhere else

Ridgeview Medical Center's radiologists dictate straight into Chartline, a tool that drafts report language as they talk and learns each doctor's usual phrasing. It was built and tuned against a specific boilerplate line for a clear head CT, the exact sentence an ER discharge checklist searches for automatically. The vendor swapped Chartline's model for a cheaper one this quarter, with no notice to hospitals already using it.

G, groups. Radiologists like Dr. Farrah Bhandal, who dictate dozens of scans a shift with a phrasing cadence they no longer think about, versus the ER staff downstream whose discharge checklist macro depends on Chartline's exact old wording showing up.
U, unequal. Barely matters on a quiet afternoon shift with time to double check. It lands hardest overnight, when a patient is waiting on a discharge decision that a checklist macro cannot clear because the wording it searches for no longer appears, even though the finding is still negative.
A, ability to contest. A patient in the ER has no way to know a phrasing change is behind their wait. Dr. Bhandal assumed it was her own dictation trailing off, not a model update, since nothing in Chartline ever told her the model had changed.
R, reduce. Ship an explicit, plain-language note the first time a radiologist's report would have used the old boilerplate phrase and does not, "standard negative-finding wording changed, the discharge macro may not recognize it yet." Lock the exact phrases downstream systems key off as fixed templates the model fills in, not free text it composes fresh each time.
D, detect. Before any Chartline update ships, run it against a frozen set of real past dictations and check whether the boilerplate phrases downstream systems depend on still come out word for word. After it ships, track the discharge-macro "impression incomplete" flag rate by radiologist, watched daily for the first two weeks.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0