ConceptAdvancedAI Opportunity & Model Strategy / Model selection from a PM lens / #11
Explain the case for supporting multiple models behind an abstraction layer.
BOUNDthe rewrite always cost about three months of one engineer. the layer cost eleven.
Skylark Events runs a live-events ticketing marketplace. Four features depend on a model today: support chat, fraud-flagging on new listings, listing-photo moderation, and search-ranking rerank. Zanele Umeh is the platform architect who has to make the case for whether these should sit behind one shared layer or stay wired straight to whichever vendor shipped first.
The direct answer
Build the abstraction layer. Every time a vendor changes pricing or deprecates a model, a hardwired integration costs Skylark about 50,000 dollars and six weeks of frozen roadmap, repeated once per dependent feature. The same event behind a shared layer costs about 1,300 dollars and an afternoon. At Skylark's rate of roughly one and a half vendor-change events a year across four features, the layer's 68,000-dollar build cost pays for itself in under a year, and every feature added after that is free.
Do this, in order
Build the shared layer before the second AI feature ships hardwired, not after the third.Why: the cost of staying hardwired multiplies with every feature added on top of it.
Standardize one request and response shape, and one place to log cost and latency per call.Why: without this, every new feature reinvents its own fragile integration from scratch.
Build one fallback rule for when a vendor is down, slow, or rate-limited.Why: an outage in one hardwired feature shouldn't require a separate incident response for every feature sharing that vendor.
Estimate the cost of the next vendor-change event before it happens, not after.Why: the real case for the layer is the scramble you'd otherwise pay for, not a preference for tidy code.
Re-check the assumption that swings the estimate most: how often vendors actually change terms.Why: at half the assumed frequency, the payback period roughly doubles, and that's worth knowing before committing budget.
Skip the layer for a single feature tied to one model with no second feature planned.Why: the build cost isn't worth paying until there's a second dependent feature to share it with.
How to answer this, stage by stage
Nobody is scoring whether you know the phrase "vendor lock-in." They're scoring whether you can put a real number on the thing you're protecting against.
Stage 1
Scope it to one real system
Say it like this
"Let's ground this in one real case. Skylark Events has four features calling models directly: support chat, fraud-flagging, photo moderation, and search rerank. That's the exact system I'd size the abstraction layer's case against."
Why this works
Stops "explain the case" from turning into an architecture opinion with no real cost attached to either side.
Stage 2
Say the structure out loud
Say it like this
"I'll run this as BOUND. Break it down, the equation before any numbers. Own the numbers, state each assumption and where it came from. Use a range, not one false-precise figure. Nail the sanity check. Direction, which assumption swings it most."
Why this works
Signals you're estimating, not guessing with confidence, which is exactly what this question is testing.
Stage 3
Break down the equation
Say it like this
"Cost of staying hardwired equals cost per call site to rewrite, times call sites per feature, times features, times vendor-change events per year. Cost of the layer equals a one-time build cost, plus a smaller yearly maintenance cost."
Why this works
Stating the equation before touching a single number is what separates an estimate from a guess with a confident tone.
Stage 4
Own the numbers
Say it like this
"I'll assume 10 call sites per feature, 40 total across four features, since that's roughly what a mid-sized integration touches. I'll assume 3 hours per call site to rewrite and retest at a blended 85 dollars an hour, plus about 40,000 dollars of delayed roadmap during the 6-week scramble. And I'll assume 1.5 vendor pricing or deprecation events a year, based on how often major model vendors have actually changed terms recently."
Why this works
Naming where each number came from is what makes the estimate defensible instead of invented on the spot.
Stage 5
Give the range and the sanity check
Say it like this
"That puts payback somewhere between 8 and 17 months depending on how often vendors actually change terms, with 11 months as the middle case. As a sanity check, one hardwired event costing 50,000 dollars is about three months of one senior engineer's fully loaded cost, which is a believable number, not an exaggerated one."
Why this works
A range plus a sanity check is what makes an estimate trustworthy instead of a single suspiciously precise figure.
Stage 6
Name the assumption that swings it most, then close
Say it like this
"The number that moves this most is how often vendors actually change terms. At half my assumed rate, payback stretches past two and a half years. I'd check that history before committing budget. But even in the slow case, the layer pays for itself eventually and every feature added after the first two is free. Build it now, before the third feature ships hardwired."
Why this works
Naming the assumption that would change the answer most is what a good estimator does and a guesser never bothers with.
Let's learn
The whiteboard is where Zanele sketches every model call Skylark Events makes in a week.
Before any of this mattered, one feature, support chat, called one model directly. It worked, it shipped fast, and nobody thought twice about wiring it straight to that vendor's exact API.
Three more features followed the same pattern over two years: fraud-flagging on new listings, photo moderation, and search-ranking rerank. Each one, on its own, was a small, sensible choice.
Four features, one shared router. Before the layer existed, each of these was its own separate wire to the same vendor.
Here's the turn: the real problem was never any single feature's integration. It's that nobody had a shared record of how many features had quietly made the same bet on the same vendor, until a deprecation notice forced everyone to add it up at once.
Four small, sensible decisions added up to one large, invisible bet that nobody had actually chosen to make.
At its worst, that bet costs 40 call sites' worth of emergency rewrites, all competing for the same handful of engineers, during the exact week a vendor's deadline is ticking down.
The choice I would take back
Skylark never kept a shared inventory of which features called which vendor directly. Each engineer wired their own feature the fastest way that shipped, and nobody paused to ask whether the next feature would do the same thing. That was fine when there was one feature. It stopped being fine the moment there were four, and nobody could see the total exposure until a deprecation notice forced everyone to count it at once.
What I would leave alone: a single feature tied to one model, with no second AI feature on the roadmap, doesn't need this layer at all. The build cost isn't worth paying for a bet nobody plans to make twice.
The lesson: the cost of an architecture decision isn't the cost of the first feature it supports. It's the cost multiplied by every feature that quietly copies the same shortcut afterward.
Now here is the same thing as a story
The short version above is what you'd say in an architecture review. Read this one for what it felt like the year the bet everyone had made separately finally came due at once.
Her name is Zanele. She's been the one architect at Skylark Events who understands every model wired into the ticketing app, for three years.
It wasn't one dramatic outage that changed her mind. It built up slowly, four features over three years, each added by a different engineer, each one a reasonable choice made in isolation, with nobody stepping back to look at the shape of all four together.
Four hardwired plugs looked like four separate decisions. They were really one decision, made four times without anyone noticing the pattern.
Then, in year three, the vendor announced it was sunsetting the exact model version all four features depended on, with a 90-day migration window.
Three years of small, separate decisions became one large bill, due all at once.
Forty call sites, across four features, all needing to be found, tested, and rewritten inside the same 90-day window, by the same small team that also had a roadmap to ship.
Knowledge spark: why do model vendors deprecate or reprice more often than other infrastructure?
Model providers retire older versions and change pricing on a much faster cycle than a database or a cloud region does, often with just months of notice, because the underlying technology itself is still moving fast. A system built assuming today's model and price hold steady is built on a much shorter shelf life than most infrastructure decisions.
The rewrite took six weeks of calendar time, not because the code was hard, but because 40 call sites meant 40 small merge conflicts with whatever else the team was shipping. Two other roadmap items paused entirely.
Five steps, and the second one, every call still hardcoded, is the one this whole estimate turns on.
We didn't pay for one broken feature. We paid, four times over, for never having built a wall between "our product" and "their API."
When each feature was first built, nobody in that planning meeting raised the shared-layer question. "Let's just call the API directly, it's one feature, we'll deal with it later," someone said each time, and each time it was a reasonable thing to say, because in isolation, it was true.
The real question was never whether any single integration was well built. It was whether Skylark had a way to see, before the third or fourth feature shipped, that the same bet was being placed again.
What the abstraction layer's 68,000-dollar build cost is made of
None of these four pieces is optional if the layer is going to actually replace all 40 hardwired call sites, not just the newest one.
Three things, and a rewrite that skips any one of them isn't really an abstraction layer, it's a wrapper around one vendor.
With the layer built after the deprecation forced the issue, the next vendor change, whenever it comes, becomes a single config update and one round of evals, about 1,300 dollars and an afternoon, instead of six weeks and 50,000 dollars spread across four features.
Payback period, by how often vendors actually change terms
Event frequency is the assumption that swings this estimate the most. Everything else moves the answer by weeks, not years.
What I'd tell myself, watching the six-week scramble unfold: four small, sensible decisions had quietly become one large bet, and nobody had ever been asked to approve the bet itself.
BOUND, mapped onto one architecture caseNot a script for building a shared layer around every single API call. BOUND is what tells you when the arithmetic actually favors it.
B
Break it down. The equation before the numbers.
Cost of hardwired equals call sites per feature, times features, times cost per site, times vendor-change events per year. Cost of the layer equals a one-time build plus yearly maintenance.
Stating the equation first is what makes this an estimate instead of a confident guess.
O
Own the numbers. State each assumption and where it came from.
10 call sites per feature, 3 hours per site at 85 dollars an hour, plus 40,000 dollars of delayed roadmap per event, and 1.5 vendor-change events a year based on recent vendor history.
Every number here is checkable against a real source, not invented to make the case look better.
U
Use a range. Not one falsely precise number.
Payback runs from 8 months at the high end of vendor-change frequency to 33 months at the low end, with 11 months as the realistic base case.
A single number here would imply a confidence about vendor behavior nobody actually has.
N
Nail the sanity check. Does it survive a smell test?
A 50,000-dollar hardwired event is about three months of one senior engineer's fully loaded cost, a believable number, not an exaggerated one built to win an argument.
If the number seemed absurd next to something familiar, that would be the signal to go back and check the assumptions.
D
Direction. Which assumption swings it most.
How often vendors actually change terms. At half the assumed rate, payback triples, from 11 months to over two and a half years.
This is the hardest step and the one that separates a good estimator from someone who just picked numbers that made their case look good.
The case for the layer isn't universal. It's a threshold, and Skylark had already crossed it twice over before anyone noticed.
The recap, one line per letter: break it down is hardwired cost as call sites times features times cost per site times events per year, own the numbers is naming where each figure came from, use a range is 8 to 33 months of payback depending on event frequency, nail the sanity check is comparing 50,000 dollars to three months of an engineer's cost, and direction is event frequency being the one assumption worth double-checking before committing budget.
And if you want to be sure it really works, try it somewhere elseSame five letters, a crop-advisory app instead of a ticketing marketplace. The features change, but the seam sits in the same place.
Farida Ojo builds AgriLoop, a farm-advisory app combining a vision model that identifies crop disease from a photo and a text model that answers a farmer's follow-up question. Mapped onto BOUND: break it down, hardwired cost equals call sites per feature times features times rewrite cost times events per year, same equation, two features instead of four. Own the numbers: 6 call sites per feature, 12 total, at a smaller 60 dollars an hour blended rate since AgriLoop is an earlier-stage team, and roughly 1 vendor-change event a year given it depends on both a vision and a text provider. Use a range: payback runs from 14 to 26 months depending on event frequency, since fewer features means a smaller yearly avoided cost. Nail the sanity check: a single event's cost, about 9,000 dollars, is under two weeks of one engineer's time, a modest but real number for a team this size. Direction: at AgriLoop's smaller scale, the number that swings it most isn't event frequency, it's whether a third feature gets added within the next year, since that's what would make the case clearly worth it now instead of marginal.
Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "it turns a repeated 50,000-dollar scramble into a 1,300-dollar config change, and it pays for itself in under a year," and stop.
Cost: no engineering budget for a full layer right now. Say so honestly, and commit to at least logging which features call which vendor, the cheapest version of the same insight.
The model got better, for real: if a vendor's new version is a straightforward upgrade with no breaking changes, that's still the moment to route it through the layer's config, not to quietly re-hardwire a call site out of habit.
Where people run it wrong.
They build the layer for its own sake, as an architecture preference, with no real cost estimate behind the case at all.
They wait until the third or fourth feature is already hardwired before ever running this arithmetic.
They present a single confident payback number instead of a range, which falls apart the moment someone asks "what if a vendor doesn't change anything for two years."
How to use it live. The moment an interviewer asks for the case for an abstraction layer, ask yourself: what did the last vendor-change event actually cost, in dollars and weeks, and how many more features are about to make that bet again? Do that arithmetic out loud, and the case makes itself.
Flashcards (tap any card to flip it)
1 · THE FRAMEWORK
What framework fits making the case for a multi-model abstraction layer?
Tap to flip
ANSWER
BOUND: break it down, own the numbers, use a range, nail the sanity check, direction. It turns an architecture opinion into real arithmetic.
2 · THE PERSON
Who is this answer about?
Tap to flip
ANSWER
Zanele Umeh, platform architect at Skylark Events, who watched four separately reasonable decisions add up to one large, invisible bet.
3 · THE EQUATION
What's the equation for the cost of staying hardwired?
Tap to flip
ANSWER
Call sites per feature, times number of features, times rewrite cost per site, times vendor-change events per year.
4 · THE OLD DECISION
What decision would you take back?
Tap to flip
ANSWER
Never keeping a shared inventory of which features called which vendor directly, so nobody could see four features had made the same bet until a deprecation notice forced everyone to count it at once.
5 · THE NUMBER
Fill in the blank: one hardwired vendor-change event cost about $___ and six weeks. The same event behind the layer costs about $___.
Tap to flip
ANSWER
About $50,000 hardwired, versus about $1,300 behind the abstraction layer.
6 · THE SANITY CHECK
How does this answer sanity-check the $50,000 hardwired cost?
Tap to flip
ANSWER
It compares it to about three months of one senior engineer's fully loaded cost, a believable figure rather than an exaggerated one.
7 · THE REPLAY
Same deprecation notice, abstraction layer already built. What changes?
Tap to flip
ANSWER
The six-week, 40-call-site scramble becomes a single config change and one round of evals, about 1,300 dollars, done in an afternoon.
8 · CROSS PRODUCT TRANSFER
Section 4 runs this again for a different product. Which one, and what stays the same?
Tap to flip
ANSWER
AgriLoop's crop-advisory app, with a vision and a text model. The scale shrinks, but the equation and the discipline of naming the swing assumption stay identical.
Check yourself Score: 0 / 0
Multiple choice
1. What single assumption swings this estimate's payback period the most?
A. The hourly rate used for engineering time.
B. The number of call sites per feature.
C. How often vendors actually change pricing or deprecate a model.
D. The size of the ticketing marketplace's total user base.
Show hint
Look at the Direction step and the horizontal bar chart of payback periods.
Show answer
C. Payback ranges from 8 months to 33 months depending only on this one assumption, far more than any other input moves the number.
True or false
2. True or false: this answer recommends building an abstraction layer for any single feature that calls a model.
True
False
Show hint
Look at "what I would leave alone" and the decision tree diagram.
Show answer
False. A single feature tied to one model with no second feature planned doesn't need the layer yet, since there's no repeated bet to protect against.
Fill in the blank
3. Fill in the blank: the abstraction layer's total one-time build cost was estimated at $___, made up of four pieces.
Show hint
Look at the waterfall chart adding up the build cost.
Show answer
$68,000. Made up of a $15k interface schema, $20k router and fallback logic, $25k migrating four existing features, and $8k of cost and latency logging.
Short answer, where it wouldn't matter
4. Describe a situation where building this abstraction layer would be a waste of effort.
Show hint
Look at "what I would leave alone."
Show answer
Model answer: A team with exactly one feature calling one model, with no second AI feature on the roadmap. There's no repeated bet yet, so there's nothing the layer would actually be protecting.
Short answer, apply it yourself
5. Think of a tool or service you rely on for more than one purpose. What would it cost you, in time, if the company behind it suddenly changed how it worked?
Show hint
Look for something you use in more than one way, the way Skylark used one vendor across four features.
Show answer
Model answer: A shared spreadsheet template used across three different school subjects: if the template's owner suddenly restructured it, updating all three uses at once would cost real hours, exactly like Skylark's 40 call sites.
Short answer, work the number
6. If Skylark only had two dependent features instead of four, would the abstraction layer still be worth building at the same 1.5-events-per-year rate?
Show hint
Halving the number of features roughly halves the avoided cost per event, which roughly doubles the payback period.
Show answer
Model answer: Payback would stretch to roughly 22 months instead of 11, since avoided cost per event would be about half as large. Still worth building eventually, but a much weaker case for doing it immediately.
Before you close the answer
Why this works
Tests whether you can turn an architecture preference into real arithmetic, with assumptions stated, a range instead of false precision, and a sanity check that would catch an inflated case.
Follow-up traps
"Isn't this just premature optimization for a small team?" Response: the arithmetic says otherwise once there are two or more model-dependent features; the layer isn't about scale, it's about how many times the same bet gets repeated.
"What if a vendor never changes anything?" Response: then payback stretches out, potentially past two years, which is exactly why the estimate uses a range and names event frequency as the assumption worth checking first, not a fixed promise.
If pressed
The layer also standardizes one place to run evals across vendors, so swapping a model isn't just a config change, it's a config change plus an automatic re-run of the same golden set every dependent feature already trusts.
From U2xAI Academy
From answering questions to owning outcomes.
A live workshop where you ship a working AI agent, defend a launch decision, and walk away with a portfolio recruiters can't wave off, not just more questions to study.