How do you handle a customer who wants their data excluded from all improvement loops?

Auberon Commerce sells recommendation and search AI to online retailers, improved by pooling anonymized clickstream and purchase data across every store on the platform. Naledi Mokoena is the enterprise AI PM who had to answer Oatley & Vance Department Stores, a top-five account, the week it asked to be excluded from every improvement loop, permanently.

How to answer this, stage by stage

Nobody is testing whether you'll say yes to the customer. They're testing whether you notice there's a second person in this story who was never asked anything.

Let's learn

Here is what happens when a request from a powerful customer exposes a gap that was never really about that customer.

Before this request, Auberon Commerce pooled clickstream and purchase data across roughly 300 retail tenants, with Oatley & Vance alone contributing about 8 percent of total training volume, enough to meaningfully shape the shared recommendation model. Every tenant had opted in by default when they signed up, back when nobody had ever asked to opt back out.

One side holds the lever. The other doesn't even know there is one.

Here's the turn: the interesting risk was never whether Auberon would say yes to Oatley & Vance. Of course it would, contractually it had to. The real risk was that the person whose actual data this was, an ordinary shopper browsing Oatley & Vance's site, was never part of this conversation at all, and honoring the request does absolutely nothing for their say in any of it going forward.

At its worst, "excluded" means only "we stopped adding new data," while months of old data keeps quietly shaping today's recommendations, and nobody at Auberon can say for certain how much influence remains, because nothing was ever built to measure or remove it.

What I would leave alone: the underlying shared-model architecture itself doesn't need to change. Pooling anonymized signal across willing tenants is a sound design; the gap was never having a clean, structural way to remove a tenant who stops being willing.

The lesson: a default that works fine when everyone agrees becomes a real liability the day even one party disagrees, and by then it's a lot more expensive to fix than it would have been to build correctly from the start.

Now here is the same thing as a story

The short version above is what you'd say defending the exclusion plan to leadership. Read this one for how a new hire's honest question exposed what "excluded" actually meant.

Naledi had run enterprise AI product at Auberon Commerce for four years, long enough to remember when Oatley & Vance's onboarding call was mostly the client asking how quickly the recommendation lift would show up.

Nobody decided, on any single day, that exclusion would be this hard. The default from launch day just never got revisited.

The request itself was polite and unambiguous: a new privacy lead at Oatley & Vance wanted their shopper data out of Auberon's shared improvement loops entirely, effective immediately, citing a tightened internal data policy. Naledi's team said yes within the day. Saying yes was never the hard part.

Four ordinary steps, and a fifth one that was always going to be a dead end, for anyone but the company holding the contract.

The engineering team built a flag to stop new Oatley & Vance data from entering the pipeline, and told the account team the exclusion was complete. Three weeks later, during onboarding, a new engineer named Priyansh asked a question in a standup that nobody answered right away: "How do we know none of their historical data is still baked into the current model's weights?"

Three dots on the same chart. Only one of them was ever actually in the room for this decision.

Nobody had a clean answer, because nobody had ever built a way to measure or remove historical influence, only a way to stop new contributions. The model kept training on fresh data from 299 other tenants, but months of Oatley & Vance's past clicks were still quietly shaping what it had already learned.

Auberon had built the first of these four. The other three were the actual promise nobody had kept yet.

Once the gap was named out loud, the team built the missing piece: a formal exclusion ledger, a process to retrain the shared model without Oatley & Vance's historical data folded in, and a quarterly report on the resulting quality cost to smaller tenants, shared honestly instead of buried in an internal metric nobody outside the team ever saw.

The four promises "excluded" should have meant from day one, written down only after someone asked the awkward question.

What none of this fixed, and what I keep coming back to: the individual shopper browsing Oatley & Vance's site during all of this never knew any of it happened. Not the original bundling, not the exclusion, not the retraining. The entire negotiation ran between two companies, over data that, in the end, was actually about them.

What I'd tell myself, hearing Priyansh's question land in that standup: the request from Oatley & Vance was never really the test. The test was whether Auberon had built something that could honor a promise like that completely, the first time, without a new hire having to notice the gap by accident.

GUARD, the case laid out plainlyNot a policy memo about data privacy. GUARD is what forces you to name the person nobody in this negotiation ever asked.

The recap, one line per letter: groups are Auberon, Oatley & Vance, and the individual shopper nobody asked. Unequal is smaller tenants absorbing a quality dip and the shopper absorbing a decision with no voice in it. Ability to contest is a company with a contract versus a person with nothing. Reduce is a real exclusion ledger with active unlearning. Detect is publishing the quality cost honestly instead of burying it.

And if you want to be sure it really works, try it somewhere elseSame five letters, a veterinary diagnostics SaaS instead of a retail platform. The subject changes from a shopper to a patient nobody can name.

Thornfield Veterinary Diagnostics sells an AI tool that suggests likely diagnoses from a pet's symptoms and test results, improved by pooling anonymized case data across clinics. Cobblestone Veterinary Group, a regional clinic chain, asked to be excluded from all improvement loops after a data-handling review by their new corporate parent. Mapped onto GUARD: groups are Thornfield, Cobblestone's leadership, and the actual pet owners whose animals' case histories are in question, never consulted at any point. Unequal says other clinics on the platform lose a small amount of diagnostic accuracy on rarer conditions once Cobblestone's caseload leaves the pool, since that chain saw an unusually high volume of a few uncommon presentations. Ability to contest: Cobblestone's leadership negotiated the exclusion directly with Thornfield's product lead, Bridget Okonkwo. The pet owner whose animal's case data is actually being discussed was never in that conversation, and has no channel to ask that their own pet's case be treated any differently at all. Reduce is the same structural fix: a per-clinic exclusion flag and active removal of historical influence, not a freeze. Detect is publishing which rare conditions saw a measurable accuracy dip after the exclusion, so other clinics know to watch more closely there, rather than discovering it the hard way on a real case.

A different industry, the same picture: one party negotiating a lever, and a subject who was never handed one at all.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "honor the request as a real per-tenant flag with active historical removal, then say plainly who never got asked at all," and stop.
Cost: no engineering time budgeted to build proper historical unlearning before the next request arrives. Say so honestly, and commit to a firm date rather than quietly leaving it half-built and calling it done.
The model got better, for real: if a newer technique makes historical unlearning cheap and near-instant, that's a legitimate reason to offer exclusion faster and more completely, not a reason to skip building the audit trail around it.

Where people run it wrong.
They treat "we said yes" as the end of the story, without checking whether yes was actually delivered in full.
They quietly absorb the quality cost to everyone else instead of measuring and disclosing it.
They never name the person whose data this is actually about, because that person isn't in the room asking for anything.

How to use it live. The moment an interviewer asks about a customer wanting their data excluded, ask yourself: who actually owns this data, the company asking, or the person the data describes, and does my answer do anything at all for the second one? Answer that, and the rest of the case follows.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0