ConceptAdvancedResponsible AI & Advanced Practice / Compliance and legal partnership / #18

What is the risk of a vendor whose compliance posture is weaker than yours?

PICK the product is Almanac Payroll, which uses a third-party AI vendor to read employees' tax documents

Almanac Payroll runs payroll for mid-size companies, and its onboarding flow uses a vendor's AI tool to read scanned tax documents, W-4s, I-9s, foreign tax forms, and pull out the numbers. Fionnuala Byrne owns that onboarding flow. She keeps the vendor's two most recent security bulletins printed and clipped inside her notebook.

The direct answer
Pre-redact the most sensitive fields, Social Security numbers, foreign tax IDs, before any document reaches the vendor, and route anything that can't be cleanly redacted through the vendor's higher, encrypted tier, even though it costs more and runs slower. The small, visible cost of doing that beats the rare, hidden cost of a breach at a vendor whose own security posture you don't control and can't fully see.
Do this, in order
  1. Pre-redact Social Security numbers and tax IDs before any document reaches the vendor.Why: data the vendor never receives can't be exposed by a vendor breach, no matter what happens on their end.
  2. Route anything that can't be cleanly redacted through the vendor's compliance-grade tier.Why: some documents are messy enough that redaction risks losing real data, and those need the stronger tier instead.
  3. Set a kill line: a fixed number of vendor incidents in a window forces every document onto the compliance tier.Why: without a stated threshold, "wait and see" quietly becomes the permanent policy.
  4. Track the vendor's own incident disclosures on a schedule, not just when a headline forces you to look.Why: a vendor's compliance posture isn't a one-time check at signing, it can get quietly weaker over time.
  5. Leave the low-sensitivity fields, name, job title, start date, on the standard tier.Why: not every field carries the same risk, and upgrading all of them would just make the whole pipeline slower for no real gain.

How to answer this, stage by stage

Six stages. A tradeoff question wants your pick first, then the reasoning, never the other way around.

Stage 1
Scope it to a real vendor relationship
Say it like this
"I'll answer this for Almanac Payroll's OCR vendor, the one that reads scanned tax documents during onboarding."
Why this works
Turns "vendor compliance risk" from an abstract worry into one real data flow with a real weak point.
Stage 2
Take your position first
Say it like this
"My pick: pre-redact the most sensitive fields before they ever reach the vendor, and pay more to route what's left through their stronger tier. I'm not switching vendors today, and I'm not ignoring the gap either."
Why this works
PICK's whole point: commit to a position before the reasoning, so it's clear you can actually decide.
Stage 3
Name who feels each kind of error
Say it like this
"The redaction and upgraded tier cost us real money and a little latency, every week, and everyone can see it on a budget line. A breach costs an employee their Social Security number being exposed, and it's invisible until it happens."
Why this works
Both sides of the tradeoff get named in real units, not just "cost versus risk" in the abstract.
Stage 4
Name the asymmetry
Say it like this
"One of these costs is small, visible, and absorbed every month without anyone noticing. The other is rare and enormous, and it lands on people who never chose this vendor at all. I'm optimizing against the second one."
Why this works
Matches the direct answer's actual reasoning, not just its conclusion.
Stage 5
Give the kill criteria
Say it like this
"If the vendor has two disclosed security incidents in any six-month window, everything moves to the compliance tier automatically, no exceptions, no re-litigating it in a meeting."
Why this works
Shows what would actually change your mind, which is what separates a real decision from stubbornness.
Stage 6
Close on one line
Say it like this
"A vendor's weaker posture isn't a reason to panic and rip them out today. It's a reason to control exactly what reaches them, and set the line in advance for when that stops being enough."
Why this works
Restates the position in one breath, so the answer closes on the decision, not a vague sense of unease.

Let's learn

Picture every tax document your payroll tool has ever processed sitting on someone else's shelf, under someone else's lock, and ask how much you actually know about that lock.

Almanac Payroll sends scanned tax documents to a third-party vendor's AI tool, which reads them and pulls out the numbers. Years ago, when Almanac's clients were mostly small, low-sensitivity accounts, the team defaulted to the vendor's cheaper standard tier, no dedicated encryption at rest, shared access logs across the vendor's other customers.

Knowledge spark: what's a subprocessor? A vendor your vendor uses. Almanac's OCR vendor stores its backups with its own separate cloud provider, one Almanac never signed a contract with directly, and never audited itself.

That default was cheap and invisible for a long time, because nothing went wrong. Then, within five weeks, the vendor disclosed two separate incidents: an access-control bug that briefly exposed a slice of client logs, and a breach at their own backup subprocessor.

Monthly cost: standard tier vs compliance-grade tier, at Almanac's volume
4,000 0 $800 Standard tier $3,600 Compliance tier
A 2,800-dollar-a-month gap sounds large in isolation. Against one exposed employee's Social Security number, it isn't.

At its worst: an employee's Social Security number sits in a vendor's backup, at a subprocessor Almanac never chose, exposed for hours before anyone outside the vendor even knows to look.

The decision I would take back We defaulted the vendor integration to its cheaper standard tier when we first signed the contract, because our client base then was small and low-sensitivity, and the price difference mattered more than the marginal risk. It stopped making sense once we onboarded larger clients with international employees, whose documents carry more sensitive identifiers than the original integration was ever sized for.

What I would leave alone: low-sensitivity fields, an employee's name, job title, start date, don't need the same treatment. Upgrading every field equally would just slow the whole pipeline down for no real reduction in risk.

The cheap tier was never really cheap. It just moved the real cost somewhere we couldn't see it until it landed.

The lesson: a vendor's compliance posture isn't a box you check once at signing. It's a number that can quietly get worse, on their end, while your own controls stay exactly the same.

Now here is the same thing as a story

The short version above is what you'd say defending this call to a board asking why costs went up. Read this one for how Fionnuala actually got there.

Every Monday, Fionnuala Byrne pulls up the vendor's status page out of habit, a small routine check that had never once turned up anything worth a second look.

The habit had been paying off in the sense that it never had to. For most of a year, documents flowed straight from Almanac's onboarding flow to the vendor's standard tier, unredacted, exactly as the integration had been built years earlier.

Hand sketched flow diagram titled Where a document goes before it's safe. Five boxes: doc uploaded, redact SSN highlighted, vendor OCR reads it, data returned, record saved.
Five steps, and the second one, for a long time, simply didn't exist.

Then, on a Monday like any other, the vendor's status page showed something new: a disclosed access-control bug, since patched, that had briefly exposed a slice of client access logs. Five weeks later, a second disclosure landed, this time about the vendor's own backup subprocessor.

Hand sketched metaphor scene titled Weighed and found lighter. Left panel, a scale icon labeled VENDOR, caption shared logs standard tier. Right panel, a box icon labeled ALMANAC, caption encrypted isolated audited.
Almanac's own controls hadn't moved at all. The vendor's had, twice, in five weeks.

Two in a row was the whole trigger. Fionnuala didn't wait for a third. She pulled every field the vendor ever touched and sorted them by what actually happens if the vendor's weaker posture is the reason it leaks.

Hand sketched comparison diagram titled The asymmetry drawn. Left panel, a box icon labeled Redaction time, caption small visible budgeted every week. Right panel, a question mark box icon labeled A leaked SSN, caption rare hidden and enormous when it lands.
One box is small and known. The other is rare, and it's shaped nothing like the first one.

She ranked every possible path by cost against risk, not just against convenience.

Hand sketched quadrant titled Ranking the vendor paths. Axes cost per document and risk if the vendor is breached. Standard tier no redaction sits top left, cheap and high risk. Compliance tier sits bottom right, expensive and low risk. Standard tier pre redacted sits lower middle.
Pre-redacting on the cheap tier turned out to sit almost as safe as the expensive tier, for a fraction of the cost.

She also went back and read what the vendor's own attestation actually promised, closely, for the first time since signing.

Hand sketched labeled parts diagram titled What a real attestation needs. Center document icon labeled Vendor Attestation, with four callouts: independent audit, named controls, breach SLA, renewal date.
The vendor's attestation had a renewal date well past due. Nobody had checked it in two years.

With the fix, Social Security numbers and foreign tax IDs get redacted before any document reaches the vendor at all, and anything too messy to redact cleanly routes through the compliance tier automatically. A second incident within six months now triggers a full move to the compliance tier for every document, no exceptions.

Hand sketched icon list titled What Fionnuala watches now. Four items: a gauge icon labeled percent of sensitive docs pre redacted before upload, a document icon labeled vendor incident disclosures per six months, a scale icon labeled cost gap between standard and compliance tier, a person icon labeled employees whose documents crossed the tier at all.
Four numbers, and the second one is the one that started this whole redesign.

The old pipeline asked whether the vendor's tool worked. The new one also asks what happens to the data on the vendor's own worst day, not just Almanac's.

I defaulted to the standard tier because the price gap mattered more back when the client base was small and low-risk. It took two of the vendor's own disclosures in five weeks, not an incident of our own, to see that a vendor's compliance posture isn't fixed at signing, it can quietly slip while your contract stays exactly the same.

PICK, drawn outNot a comparison table. PICK is what forces you to commit to a position, then find the one cost that actually changes behavior.

P
Position. The pick, before the reasoning.
Pre-redact the most sensitive fields before they reach the vendor, and route the rest through the compliance tier, without switching vendors today.
Commits to a real decision instead of hedging with "it depends."
I
Impact. Who feels each error.
Engineering and budget feel the redaction cost, in dollars and latency, every week. An employee feels a breach, in a leaked Social Security number, rarely but severely.
Names both sides in real units, not just "cost versus risk" as an abstraction.
C
Cost asymmetry. The hardest step.
The redaction cost is small, visible, and absorbed monthly. The breach cost is rare, hidden, and enormous when it lands, on someone who never chose this vendor.
The whole tradeoff turns on this. Optimize against the hidden, expensive side.
K
Kill criteria. What would flip the pick.
Two disclosed vendor incidents within six months moves every document to the compliance tier, automatically, no meeting required.
Separates a real decision from stubbornness, since it names in advance what would change it.
Vendor's disclosed incidents, cumulative, against the kill line
3 0 kill line: 2 in 6mo 1st incident 2nd, crosses line
The two incidents landed five weeks apart. The kill line existed by the time the second one did, and the fix triggered without a debate.

The recap, one line per letter: position is pre-redact and route through the compliance tier rather than switching vendors immediately, impact is engineering's monthly cost against an employee's rare but severe exposure, cost asymmetry is optimizing against the hidden, expensive side, and kill criteria is two incidents in six months forcing the stronger tier automatically.

And if you want to be sure it really works, try it somewhere elseSame four letters, a travel-booking marketplace instead of payroll. This time the vendor screens for fraud, not tax documents, and the tradeoff runs the same shape anyway.

Farrow Travel uses a third-party AI vendor to screen bookings for payment fraud before confirming a reservation. Kwabena Sarpong manages that fraud-screening integration.

Mapped onto PICK: position is keeping the current vendor but stopping any full card number from ever reaching their basic-tier endpoint, using a tokenized reference instead, while routing only the flagged, harder cases through the vendor's stronger, audited endpoint. Impact is the token-only approach adding a small amount of screening latency, felt by customers waiting an extra second at checkout, against a card-data breach that would be felt by customers who never see the vendor's name at all. Cost asymmetry is that the latency cost is visible and constant, while a card-data breach is rare and catastrophic, both for customers and for Farrow's own ability to keep accepting payments at all. Kill criteria is a single disclosed card-data incident, of any size, immediately moving all bookings to the tokenized path, since payment data doesn't get the two-strikes leeway a lower-sensitivity field might.

Hand sketched quadrant reused to represent ranking fraud screening vendor paths at a travel marketplace by cost and risk.
A different vendor, a different kind of sensitive field, and the same corner of the chart still wins.

Swap the trigger and it still runs.
Speed: an interviewer caps you at a minute. Say "control what reaches the vendor, upgrade what you can't control, and set the line in advance for when that's not enough," and stop.
Cost: finance says the compliance tier is too expensive to roll out broadly. Start with just the highest-sensitivity fields, since that's where almost all the real risk concentrates anyway.
The model gets better, for real: if the vendor's own OCR accuracy improves, that's no reason to relax the redaction. A more accurate reader of unredacted Social Security numbers is still an unredacted Social Security number sitting on someone else's server.

Where people run it wrong.
They treat a vendor's compliance certification as a permanent fact checked once at signing, instead of a status that can quietly change.
They panic and rip out a vendor immediately after one incident, when a real kill line, stated in advance, is usually the steadier answer.
They upgrade every field to the expensive tier equally, instead of finding the small set of fields that actually carry the real risk.

How to use it live. When this question comes up, ask yourself what specifically reaches the weaker vendor, not just how weak the vendor is in general. The fix usually lives in controlling that, not in the vendor relationship itself.

Flashcards (tap any card to flip it)

1 · THE FLIP FAMILY
What flip family is this?
Tap to flip
ANSWER
Input flip: the team stops sending raw, unredacted documents to the vendor and starts pre-redacting sensitive fields first, a real change in what gets fed to the system.
2 · THE PEOPLE
Who is this answer about?
Tap to flip
ANSWER
Fionnuala Byrne, who owns Almanac Payroll's onboarding flow and checks the vendor's status page every Monday out of habit.
3 · THE HABIT
What habit paid off without ever having to prove itself?
Tap to flip
ANSWER
Fionnuala's weekly check of the vendor's status page, which had never turned up anything, until it did.
4 · THE FLIP, IN THIS STORY
What's the two-setting switch here?
Tap to flip
ANSWER
Sending documents to the vendor exactly as scanned versus pre-redacting sensitive fields first. Once the second vendor incident landed, there was no going back to the first setting.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Defaulting the vendor integration to the cheaper standard tier at signing, since it made sense when Almanac's client base was small and low-sensitivity.
6 · THE NUMBER
Fill in the blank: the compliance-grade tier costs about ___ more per month than the standard tier, at Almanac's volume.
Tap to flip
ANSWER
2,800 dollars (800 versus 3,600). Small against the cost of one exposed employee's Social Security number.
7 · THE REPLAY
Same vendor, two incidents in five weeks, redesigned pipeline. What changes?
Tap to flip
ANSWER
The kill line already existed by the second incident, so every document moves to the compliance tier automatically, without a meeting to debate it.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this again for a different product. Which product, and what's the kill line there?
Tap to flip
ANSWER
Farrow Travel's fraud-screening vendor. There, a single disclosed card-data incident, of any size, immediately moves all bookings to the tokenized path.

Check yourself Score: 0 / 0

Fill in the blank
1. Fill in the blank: this answer's kill criteria is ___ disclosed vendor incidents within six months.
Show hint
Look at the K step and the line chart with the kill line marked.
Show answer
Two. Once the second incident landed within five weeks of the first, the pre-stated threshold triggered without needing a new discussion.
True or false
2. True or false: this answer's position is to switch away from the vendor immediately.
  • True
  • False
Show hint
Look at the P step, position, in the PICK recap.
Show answer
False. The position is to control what reaches the vendor and upgrade the tier for sensitive fields, not to switch vendors on day one.
Multiple choice
3. Why does this answer optimize against the breach risk rather than the redaction cost?
  • A. Redaction is technically impossible to build.
  • B. The redaction cost is small and visible every month, while a breach is rare, hidden, and lands on someone who never chose the vendor.
  • C. The vendor requires it contractually.
  • D. Breach risk is actually cheaper to absorb than the redaction cost.
Show hint
Look at the C step, cost asymmetry, in the recap.
Show answer
B. PICK's core move is finding which side of a tradeoff is hidden and expensive, then optimizing against that one specifically.
Short answer, where it wouldn't matter
4. Name a field in this pipeline where upgrading to the compliance tier wouldn't be worth it.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: Low-sensitivity fields like an employee's name, job title, or start date. Upgrading those wouldn't meaningfully reduce risk, just slow the pipeline down.
Short answer, apply it yourself
5. Pick a service you use that relies on a third-party vendor behind the scenes. What's the one piece of your data you'd most want that vendor to never actually see?
Show hint
Think about payment processors, ID verification tools, or customer support chat vendors.
Show answer
Model answer: Most people land on a specific number, a card number, an ID number, a password, rather than a vague "all my data," which is exactly the kind of specific field this answer targets for redaction.
Before you close the answer
Why this works
Tests whether you can name a real cost asymmetry instead of just saying "vendors are risky," and whether you'll commit to a specific position with a stated threshold for changing it, rather than an open-ended promise to "monitor the situation."
Follow-up traps
"Isn't pre-redaction risky if you accidentally strip real data the vendor needed?" Response: that's exactly why messy, hard-to-redact documents route through the compliance tier instead of being force-redacted, rather than treating redaction as the only fix.

"Why not just switch vendors right away instead of adding your own redaction layer?" Response: a vendor switch takes months to qualify and migrate safely, while redaction can ship in weeks, so it's the faster real protection while a longer-term vendor decision gets made properly.
If pressed
Almanac's real redaction step runs as a separate, smaller model trained only to locate and mask specific identifier patterns, kept intentionally dumb and auditable, rather than reusing the same general model that reads the rest of the document.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more