ConceptIntermediateAI Opportunity & Model Strategy / Evaluating AI vendors as a buyer / #12

Describe the integration questions that determine real total cost.

ORDER the bill a neighboring library system never saw coming, until it needed to leave

Endsleigh County Library Consortium shares one catalog system across a dozen small branches. Ansel Petrosky is the systems librarian evaluating Cataly, a vendor that auto-generates subject tags and metadata for new acquisitions. A peer consortium's own costly exit from a similar vendor is what changed how Ansel asked his questions.

The direct answer
Rank integration questions by what's hardest to undo, not by what's cheapest to ask first. Ask before anything else whether you can export your data back out in a standard format, and at what cost, since that decision is nearly impossible to reverse once you've committed. Only after that do migration labor, recurring sync fees, and who owns the generated metadata become worth pricing in.
Do this, in order
  1. Ask whether you can export your data back out in a standard format, and at what cost, before anything else.Why: this is the hardest decision to undo, and it decides whether every other cost is temporary or permanent.
  2. Price the one-time migration-in labor at your own real record volume, not the vendor's example customer.Why: a small pilot at your own records tells you the true per-record cost before you commit to all of them.
  3. Ask if there's a recurring sync or API cost beyond the license fee.Why: a fee hidden inside how the integration works can dwarf the license line over a few years.
  4. Confirm who owns the AI-generated metadata once it's created.Why: ownership decides whether switching vendors later costs a simple export or a full redo.
  5. Run a small, cheap pilot before committing to full migration.Why: it's the one way to learn the real numbers above before the biggest, least reversible cost lands.

How to answer this, stage by stage

Nobody is scoring whether you can list integration questions. They're scoring whether you can rank them by which one you can't take back.

Stage 1
Scope it to one real integration decision
Say it like this
"I'll answer this for Endsleigh County Library Consortium, evaluating Cataly's AI cataloguing tool, not vendor integrations in general."
Why this works
Keeps "integration costs" from turning into a generic procurement checklist.
Stage 2
Say your structure out loud
Say it like this
"I'll use ORDER. Outcome, what we're really pricing. Reversibility, what's hardest to undo. Dependency, what unblocks what. Evidence, what we can learn cheaply first. Rank, the actual order."
Why this works
Signals a repeatable ranking method instead of a random list of concerns.
Stage 3
Give the one question that matters most
Say it like this
"Before anything else: can we get our data back out in a standard format, and what would that cost? Everything else is easier to reverse than that one."
Why this works
This is the direct answer, said the way you'd actually say it out loud.
Stage 4
Prove it with the real numbers
Say it like this
"The license reads fifteen thousand a year. Migrating in costs forty two thousand once. Migrating back out, if we ever needed to, was quoted at fifty eight thousand."
Why this works
Turns "read the fine print" into one specific, checkable set of numbers.
Stage 5
Say what you'd still leave alone
Say it like this
"For a tool we could swap out in an afternoon, like a scheduling widget, I wouldn't run any of this. The caution is for decisions that lock us in."
Why this works
Shows judgment instead of applying this same rigor to every vendor decision equally.
Stage 6
Close on the one line
Say it like this
"Price the exit before you price the entry. A license fee that looks cheap can sit on top of a lock-in that isn't."
Why this works
Restates the direct answer in one breath, ready for a live follow-up.

Let's learn

Say a library consortium wants an AI tool that reads a new book's cover and table of contents and writes its subject tags automatically, instead of a librarian typing them in by hand.

Before Cataly, cataloguing a new item took a librarian about twenty minutes: scan the cover, write the subject tags, enter the record, file it in the shared system. Across all twelve branches, that's roughly eighteen thousand new items a year, plus a backlog of thirty four hundred waiting to be catalogued.

Hand sketched flow diagram titled Cataloguing a new book, before Cataly. Five boxes in sequence: Scan the cover, Write the subject tags highlighted, Enter the MARC record, File it in the ILS, Shelve the book.
This is the job Cataly is meant to speed up. Nothing about it names what happens if the consortium ever wants to leave.

Cataly's pitch cuts that to about three minutes an item, and its license reads fifteen thousand dollars a year for the whole consortium, a number that looked easy to approve on its own.

The real question was never whether fifteen thousand dollars a year was affordable. It was what that number was quietly sitting on top of.
What the license line hides, in year one alone
60,000 30,000 0 15,000 Quoted license 57,000 Real year-one total
The gap is the one-time migration-in labor nobody priced until Ansel asked for it directly.

At its worst, signing on a license number alone, with no answer on data portability, risks the consortium discovering years later that leaving costs more than joining ever did, at a moment when leaving is exactly what it needs to do.

The choice I would take back Endsleigh's old vendor-review checklist only ever asked about the license fee and the time saved. That was fine for tools that stored nothing permanent. It stopped being fine the moment a vendor's own proprietary tagging schema became the only place two hundred forty thousand catalog records lived.

What I would leave alone: a tool the consortium could swap out in an afternoon, like a simple room-booking widget for meeting spaces, doesn't need this level of scrutiny. The caution here is for decisions that are genuinely hard to reverse, not every purchase.

The lesson: a license fee is the smallest number in the decision. The real cost lives in the questions nobody thought to price until it was almost too late to ask them cheaply.

Now here is the same thing as a story

The short version above is what you'd say defending this evaluation to Endsleigh's consortium board. Read this one for how the real questions actually got asked.

Ansel Petrosky had managed Endsleigh's shared catalog system for six years, patient enough to read a vendor contract's whole appendix before anyone else on the board bothered to open it.

Hand sketched comparison titled Two doors, two kinds of decision. Left, a green circle icon labeled Reversible, caption swings both ways, cheap to undo. Right, a red-orange scale icon labeled Locked in, caption bolted shut once you commit.
Most of Cataly's pitch deck was about the left door. The real risk was hiding behind the right one.

Cataly's sales team led every conversation with the license number and the time saved per item. Both were real. Neither one mentioned what happened to two hundred forty thousand existing catalog records once they moved into Cataly's own tagging system.

Knowledge spark: what does "data portability" mean for a library system? It means whether your catalog records can move to a different system later, in a standard format everyone can read, or whether they only make sense inside the one tool that created them. A record that only Cataly can read isn't really your record anymore. It's Cataly's, with your books listed in it.

Word reached Ansel from a systems librarian at a neighboring county consortium, who'd adopted a similar AI cataloguing vendor two years earlier and later needed to switch to a new shared catalog system entirely. Getting their own records back out, in a format the new system could read, cost them ninety five thousand dollars, more than six years of the original license fee combined.

Hand sketched decision tree titled Which decision to price first. Root, evaluating an integration cost. Three branches: hard to undo later leads to price this first, easy to reverse leads to price this last, unclear yet leads to run a small pilot.
Ansel realized he'd been pricing Cataly's decisions in exactly the wrong order.

Ansel asked Cataly one question before any other: could Endsleigh export its tagged records back into standard MARC format, and at what cost. The vendor confirmed it was possible, through a paid migration service, quoted at fifty eight thousand dollars, worse than the forty two thousand dollar cost of migrating in, since Cataly's own export tool only handled a small fraction of its proprietary tag types cleanly.

The exit was never free. It just hadn't been asked about yet, and asking about it changed which number actually mattered.

He had considered simply trusting Cataly's promise that "standard export is always available" without a real quote attached. He pushed for the number instead, since a promise with no price behind it isn't a real answer, it's a sales line.

Hand sketched labeled parts diagram titled What the sticker price never shows. A document icon at center labeled License, 15,000 a year, with four callouts: migration-in labor, exit and export cost, recurring sync fees, who owns the tags.
Four questions, none of them on the pricing page. All four decided the real cost.

Ansel also found a recurring cost buried in the integration details: Cataly charged per record for its ongoing sync with Endsleigh's shared system, and the vendor's own roadmap suggested a full re-sync of the whole catalog roughly every two years as its tagging model improved, a cost that never appeared on the license page at all.

Hand sketched timeline titled The real plan, before full migration, ask the exit question emphasized. Ask the exit question, caption before signing anything. Run a 500 record pilot, caption learn the real numbers. Price the full migration, caption using pilot data. Sign, with exit terms set, caption not discovered later.
Endsleigh ran a five hundred record pilot before committing the other two hundred thirty nine thousand five hundred.

Endsleigh ran a small pilot first, migrating five hundred records to learn the true per-record labor cost, then negotiated a capped exit-migration price into the contract itself, before signing anything for the full two hundred forty thousand.

ORDER, in five movesNot a checklist of vendor questions. ORDER is what tells you which one to price first.

O
Outcome. What every integration question is really deciding.
Not "is the license affordable," but "what's the real total cost over the life of this system, including getting out."
Without naming this, ranking the questions is just opinion.
R
Reversibility. Which decision is hardest to undo.
Whether records can export back to a standard format, since a locked-in schema can't be walked back once two hundred forty thousand records depend on it.
This is the hardest step, and the whole ranking turns on it.
D
Dependency. What unblocks what.
You can't price real migration labor until you know whether Cataly reads the existing system's records natively or needs a custom transform.
Some costs are forced by reality, not a judgment call, and this is one of them.
E
Evidence. What you can learn cheaply first.
A five hundred record pilot, before committing all two hundred forty thousand, revealed the true per-record labor cost for almost nothing.
Buys real numbers before the biggest, least reversible cost ever lands.
R
Rank. State the order and defend the top pick.
Exit cost first, migration-in labor second, recurring sync fees third, metadata ownership fourth.
The order isn't arbitrary: it follows directly from what's hardest to undo.
Hand sketched icon list titled The integration questions, in order. Can we export our data back out. What's real migration labor, at our volume. Is there a recurring sync fee. Who owns the generated metadata.
This order isn't a preference. It follows directly from what's hardest to undo.

The recap, one line per letter: outcome is naming the real total cost including the exit, reversibility is the exportability question that's nearly impossible to undo, dependency is needing the technical answer before pricing labor, evidence is a small pilot revealing the real numbers cheaply, and rank is exit cost first, then migration, sync fees, and ownership.

And if you want to be sure it really works, try it somewhere elseSame five letters, a veterinary clinic network instead of a library consortium. A different dependency breaks the second ranking.

Foxhollow Veterinary Group runs eight clinics sharing one patient record system. Rosalind Kemp, its operations director, was evaluating VitaTag, a vendor offering AI-generated summaries and diagnostic tags for patient visit notes. Mapped onto ORDER: outcome is the real cost of the tool over its contract life, not just its per-visit fee. Reversibility is again the exportability question, whether VitaTag's generated diagnostic tags can move to a different veterinary record system later in a standard format. Dependency here breaks differently: Foxhollow couldn't price migration labor at all until it knew whether VitaTag's tags mapped to the clinic's existing diagnostic-code standard, a mapping VitaTag itself hadn't fully built yet, which meant the true integration cost was unknowable until VitaTag finished its own unfinished work. Evidence is asking VitaTag to run its tagging on twenty real patient records first, to see how many tags needed manual correction. Rank puts the mapping-dependency question first here, ahead of exit cost, since nothing else could even be estimated until that one technical answer existed.

Hand sketched metaphor scene titled Whose language your data ends up speaking. Left, a red-orange box icon labeled Vendor format, caption only their tool reads it. Right, a green document icon labeled Standard format, caption any tool can read it.
Foxhollow faced the same two doors Endsleigh did, just with a diagnostic code standard instead of MARC.
Cataly's cumulative cost over five years, with the exit-fee cliff marked
200k 100k 0 Year 5 Year 1 57k 117k +58k if you leave
The steady climbing line is what everyone budgets for. The dashed jump at the top is the number nobody asked about first.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "price the exit before you price the entry, since that's the decision you can't take back," and stop.
Cost: there's no time to negotiate a capped exit fee before a decision is due. Say so honestly, and at minimum get the vendor's real export cost in writing before signing, rather than skipping the question.
The model gets better, for real: if Cataly's next release ships a clean, free, standards-based export tool, that's still worth verifying on a real sample before trusting the license number alone.

Where people run it wrong.
They compare vendors by license fee alone, without asking what happens to their own existing data.
They accept a vendor's word on data portability without a number attached to it.
They price migration-in labor from a demo, instead of running a small pilot at their own real volume first.

How to use it live. The moment someone asks about integration costs, ask back: which of these decisions can we actually undo later, and which one locks us in? Price the one that locks you in first.

Flashcards (tap any card to flip it)

1 · THE FRAMEWORK
What framework fits a prioritization-shaped question like ranking integration cost questions?
Tap to flip
ANSWER
ORDER: outcome, reversibility, dependency, evidence, rank. Built for ranking by what's hardest to undo, not by what's easiest to ask.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Ansel Petrosky, systems librarian at Endsleigh County Library Consortium, who read the whole vendor contract appendix before anyone else did.
3 · THE HARDEST QUESTION
What's the single hardest-to-undo integration question, per the R step?
Tap to flip
ANSWER
Whether your data can export back out in a standard format, and at what cost. Everything else can be renegotiated more easily than this.
4 · THE GAP
What did the sticker price actually hide?
Tap to flip
ANSWER
A 15,000 dollar license sat on top of a 42,000 dollar migration-in cost and a 58,000 dollar potential exit cost, none shown on the pricing page.
5 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
A vendor-review checklist that only ever priced the license fee and time saved, never the cost of leaving.
6 · THE NUMBER
Fill in the blank: migrating out of Cataly, if ever needed, was quoted at about ___ dollars, more than the cost of migrating in.
Tap to flip
ANSWER
58,000 dollars, against 42,000 to migrate in, because Cataly's own export tool only handled a fraction of its tag types cleanly.
7 · THE REPLAY
Same vendor pitch, but Ansel asks the exit question before the license question. What changes?
Tap to flip
ANSWER
Endsleigh negotiates a capped exit-migration price into the contract itself, before signing, instead of discovering the real number only if it ever needs to leave.
8 · CROSS PRODUCT TRANSFER
Section 4 answers this same question again for a different organization. Which one, and what's different about its ranking?
Tap to flip
ANSWER
Foxhollow Veterinary Group's VitaTag. Its ranking puts an unfinished diagnostic-code mapping first, ahead of exit cost, since nothing else could be priced until that existed.

Check yourself Score: 0 / 0

True or false
1. True or false: the fifteen-thousand-dollar license fee was the biggest real cost in Endsleigh's first year with Cataly.
  • True
  • False
Show hint
Look at the first bar chart.
Show answer
False. The real first-year total, including one-time migration-in labor, came to 57,000 dollars.
Multiple choice
2. Why does exportability rank first among the integration questions here?
  • A. It's the cheapest question to ask.
  • B. Vendors are legally required to answer it first.
  • C. It's the hardest decision to reverse once you've committed.
  • D. It determines the color scheme of the new system.
Show hint
Look at the R step, reversibility, in the ORDER recap.
Show answer
C. Migration labor and sync fees can be renegotiated. Getting locked into a proprietary format usually can't be undone cheaply.
Fill in the blank
3. Fill in the blank: migrating existing records into Cataly's schema was quoted at 42,000 dollars, but migrating back out was quoted at ___ dollars.
Show hint
Look at the flashcard about the number.
Show answer
58,000 dollars. Higher than migrating in, since the vendor's own export tool couldn't handle its proprietary tags cleanly.
Short answer, where it wouldn't matter
4. Name a kind of purchase at Endsleigh where this level of integration scrutiny would be overkill.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: A tool that could be swapped out in an afternoon, like a simple room-booking widget, with nothing permanent stored inside it.
Short answer, apply it yourself
5. Think of a subscription or tool you or your team depends on. What would it cost to get your own data completely out of it?
Show hint
Ask whether you've ever actually tried exporting everything, or just assumed you could.
Show answer
Model answer: Most people haven't priced this at all, which is exactly the gap this answer is pointing at.
Short answer, name the reversal
6. What's different about how Foxhollow Veterinary Group had to rank its integration questions, compared to Endsleigh?
Show hint
Look at Section 4's dependency step.
Show answer
Model answer: Foxhollow had to rank an unfinished diagnostic-code mapping first, ahead of exit cost, since nothing else could even be estimated until that mapping existed.
Before you close the answer
Why this works
Tests whether you can rank integration questions by what's actually irreversible, rather than reciting a generic list of things to "ask a vendor about."
Follow-up traps
"Isn't asking about exit cost before you've even started a bit paranoid?" Response: no, it's the one question that's cheap to ask now and nearly impossible to answer honestly once you're already two hundred forty thousand records deep.

"What if the vendor won't give a real exit-cost number upfront?" Response: that refusal is itself useful information, and it's worth pricing the unknown risk into the contract rather than skipping the question entirely.
If pressed
Endsleigh's final contract capped Cataly's future export fee at a fixed rate per record, tied to today's dollar figure, so a future price hike on the exit itself couldn't quietly change the real cost of the decision after signing.
From U2xAI Academy

From answering questions to owning outcomes.

A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.

  • A live AI agent you actually shipped
  • A launch decision you can defend under pressure
  • An interview-ready portfolio, not more flashcards
Know more