What documentation should accompany a model deployed in a regulated context?

Palterfen is an online retailer. Palisade Pay is the model built into its checkout that approves or declines a shopper's buy-now-pay-later request in under a second. Dashiell Kettner is the product manager who owns Palisade Pay, and the documentation that has to exist alongside it, since consumer credit decisions are a regulated space.

How to answer this, stage by stageSix moves. Say your structure early, since "documentation" sounds like a checklist question until you ground it in one shopper.

Let's learn

What happens the first time a regulator asks why one specific shopper, by name, got declined eight months ago?

Palterfen built Palisade Pay to approve or decline a buy-now-pay-later request the instant a shopper hits checkout, using income signals, order history, and a few dozen other features, no human in the loop for the vast majority of decisions.

At launch, Palisade Pay had exactly one document: a two-page model card, written the week before ship, describing the version that had just passed its pre-launch eval.

At its worst, a version of Palisade Pay quietly retrained and shipped in March, using an updated income-verification signal, with no note anywhere linking that version to the decisions it was making. Nobody thought to update the two-page card. It looked the same as it always had.

What I would leave alone: a live, public, per-decision dashboard showing every score in real time isn't worth building on day one. It answers a question almost nobody is actually asking, and it's a lot of engineering effort pointed at the wrong problem.

The lesson: documentation for a regulated model isn't a one-time deliverable you file away. It's a system that has to update every time the thing it's describing does, or it quietly stops being true.

Now here is the same thing as a story

The short version above is what you'd say defending this design in a compliance review. Read this one for how the gap actually got found.

Dashiell Kettner keeps a printed model card clipped inside a binder on his desk, the same one from Palisade Pay's launch eighteen months ago, a little sun-faded at the edges from sitting near the window.

The whole decision happens in under a second, with nobody in the loop. That's exactly why the record around it has to do more work, not less.

For most of a year, that binder was enough. Regulators asked general questions, about the training data, about overall approval rates, and the two-page card answered them fine.

Then Palisade Pay's engineering team shipped a quiet retrain in March, swapping in an updated income-verification signal meant to catch a category of fraud that had been slipping through. Nobody flagged it as a documentation event. It felt like ordinary maintenance.

Same declined shopper, same regulator's question. The only difference is whether a record was built to answer it.

Eight months later, a state regulator's routine review pulled a sample of March declines and asked Palterfen to explain the reasoning behind one of them, a shopper declined for a request just under Palisade Pay's usual approval line.

Dashiell's team spent three weeks just confirming which model version had actually made that call, since nothing in the launch card said when the March retrain happened, what changed, or which decisions it covered.

A record that only opens once, at launch, has no second milestone to update at. The quarterly re-check is the piece that was missing.

With the living decision record in place, that same March decline now takes about a minute to explain: the exact model version pinned to the decision, the eval set it had just passed, the plain-language reason for the decline, and the appeal path the shopper was actually offered.

Four fields. Every one of them was the exact thing the three-week scramble was missing.

The old model card described the system once, at the moment it was proudest of itself. The new record describes every decision, at the moment it was actually made.

I treated the card as a launch deliverable because that's when everyone was in the room asking for it. It took a three-week scramble to answer one regulator's question about one shopper to see that a model that keeps changing needs a record that keeps changing with it.

SPARK, the record as a designed anchorNot a compliance form. SPARK is what makes the record something the team actually keeps current instead of writing once and forgetting.

All three are reasonable things to eventually build. None of them answer "why was this one shopper declined," which is the question that actually shows up first.

The recap, one line per letter: situation is the sub-second, human-free checkout decision, payoff is anyone answering "why" in minutes, anchor is the living decision record with its four fields, risk is the quiet retrain nobody logs, and keep out is the dashboard and the appeals team, both real, both later.

And if you want to be sure it really works, try it somewhere elseSame five letters, a textile mill instead of a checkout. A completely different industry, and here the anchor isn't a decision record, it's a defect log tied to a safety certification.

Corrigan Mills uses an AI vision model to flag defective batches of children's-clothing fabric before certification, since flammability and choking-hazard standards are regulated. Greta Manzano manages the model, and flags surface on a shared tablet three inspectors pass between them on the floor.

Mapped onto SPARK: situation is a fabric roll moving past a camera at full line speed, with an inspector glancing at a flag rather than checking every inch by hand. Payoff is an inspector trusting a flag enough to pull only the flagged sections, not re-checking the whole roll out of habit. Anchor is a per-batch certification record: which model version flagged it, which physical defect category triggered the flag, and a photo of the actual flagged section, all attached to the batch's certification paperwork before it ships. Risk is a model update that changes what counts as a flaggable defect without anyone updating what the certification record claims it's checking for. Keep out is a live company-wide defect dashboard, since a regulator asks about one certified batch at a time, not a running average across every roll.

Corrigan Mills needs the same top-right pieces as Palisade Pay, just renamed: model version pin becomes flag-version pin, reason codes become defect-category codes.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "a living decision record, pinned per decision, with the model version, the eval set, the reason, and an appeal path," and stop.
Cost: there's no headcount this quarter to build automated record generation. Say so honestly, and start with a required manual log entry per model change, even a spreadsheet row, since a slow record beats no record.
The model gets better, for real: if Palisade Pay's overall approval accuracy improves, that's still not a reason to skip logging the version, a rarer decline can still be the one a regulator picks to ask about.

Where people run it wrong.
They write a thorough model card once, at launch, and treat "documentation" as a box already checked forever.
They log the score but not the reason, so the record answers "what" and never "why."
They build an impressive live dashboard nobody asked for instead of the four boring fields a regulator actually needs.

How to use it live. When someone asks what documentation should accompany a regulated model, don't list artifacts. Ask yourself one question first: could this record answer, for one specific decision made months ago, exactly why it happened. If it can't, it isn't documentation yet, it's a launch souvenir.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0