What non-quality factors should influence model choice: name four.

Ampersand Bank runs a chat support desk for its banking app. A model answers routine questions, balance checks, refund status, a password reset, before a human ever opens the ticket. Ingrid Cassaday leads support operations, and it's her job to say which model the bank commits to, and for how long.

How to answer this, stage by stage

Nobody is scoring whether you can list four buzzwords. They're scoring whether your order would have caught the block a naive ranking misses.

Let's learn

Say we build a support chat that answers routine banking questions before a human ever sees the ticket.

Before it, a support team handled every one of 10,000 monthly tickets by hand, about six minutes each, checking balances, resetting passwords, explaining a pending refund.

With the chat model live, most tickets get answered automatically. Ampersand picked one model tier, the most capable one, for every ticket, regardless of how simple the question was.

A vending machine charges the same for everything. A support desk doesn't have to.

Here's the turn: the real problem was never just which model answers a ticket best. It's that quality was the only factor anyone ranked on purpose. The other three, lock-in, uptime, and cost, rode along on whatever the first vendor happened to offer.

At its worst, that oversight locks you into a contract you can't exit, at a spend floor higher than what you actually need, agreed to before anyone modeled how fast volume would grow.

What I would leave alone: for a small internal pilot used by 50 analysts a month, this whole ranking barely matters. The spend is trivial and switching costs nothing, so there's no lock-in worth scrutinizing yet.

The lesson: the factor that's hardest to notice is exactly the one that should rank first, not last. Cost announces itself on a monthly bill. Lock-in doesn't announce itself until you try to leave.

Now here is the same thing as a story

The short version above is what you'd say in a vendor review. Read this one for what it felt like the quarter a colleague's remark turned an easy cost fix into a much bigger conversation.

Every Monday, Ingrid pulls last week's escalation queue and reads every ticket the model closed on its own.

For the first few months, chat volume was small, about 10,000 tickets, and the bill barely registered next to everything else in the budget. Nobody thought about cost, because there was nothing yet to notice.

Four factors, and only one of them shows up on a monthly invoice.

Volume grew steadily: 10,000 tickets, then 16,000, then 22,000, then 29,000. By month five it had reached 34,000, and the bill, still one flat rate for every ticket, had grown right alongside it.

In a finance review that month, a colleague asked the obvious question: "Why are we still paying the priciest tier for every ticket, even a routine password reset?"

Ingrid built the obvious fix: route the 70 percent of tickets that were routine to a cheaper model tier, keep the complex 30 percent on the premium tier. On paper, at 40,000 tickets a month, that dropped the bill from 5,600 dollars to 2,240, a savings of 3,360 dollars every month.

When she brought the plan to finance to schedule the switch, they found the contract Ampersand signed at launch, back at 10,000 tickets a month, included a 4,000-dollar monthly minimum spend, locked for three years. The tiering plan worked. The bill still couldn't legally drop below the floor.

The colleague's remark in month five triggered the fix. The contract that capped the fix was signed in month one.

When the vendor was chosen at launch, nobody suggested reading the exit terms closely. "We already use this vendor for the fraud-copy tool, let's just extend it," someone said in that meeting, and it felt efficient, not risky, since volume was small and the minimum spend looked easy to clear.

The real question was never which model tier answered a ticket best. It was whether Ampersand had noticed, before signing anything, that it was giving up the ability to act on cost at all for three years.

Cost is the closest, easiest problem. Lock-in is the farthest, hardest one. Rank by the second axis, not the first.

Most of the time, nothing is on fire yet. That's exactly when lock-in and uptime deserve the first look, not the last.

With lock-in ranked first from the start, Ampersand would have negotiated a volume-based ramp instead of a flat three-year minimum. The tiering plan would have saved the full 3,360 dollars starting the very next billing cycle, not the 1,600 the floor allowed.

What I'd tell myself, the day finance found that clause: the number everyone noticed first was never the one that decided whether the fix would work.

ORDER, mapped onto one vendor decisionNot a script for always distrusting cost. ORDER is what tells you when a quieter factor outranks it.

The recap, one line per letter: outcome is sustainable support quality as volume triples, reversibility is vendor lock-in outranking cost precisely because it's hardest to undo, dependency is needing to clear lock-in before a cost fix can even take effect, evidence is a contract re-read that would have cost an afternoon, and rank is lock-in first, cost last.

And if you want to be sure it really works, try it somewhere elseSame five letters, a waste-collection dispatcher instead of a bank. The invoice looks different, but the seam sits in the same place.

Cormac Deighan schedules pickup routes for Verge Waste Solutions, choosing a model to answer resident questions about missed collections and bin swaps. Mapped onto ORDER: outcome is keeping dispatcher headcount flat as call volume grows through a city contract expansion. Reversibility: a single-region hosting requirement in one vendor's contract, agreed to for a launch-week discount, turns out to be the hardest thing to undo once Verge expands into a second state with its own data rules. Dependency: Verge can't finalize its cost-per-call model until it knows whether it's even allowed to route calls through a cheaper, out-of-region provider. Evidence: a half-day legal review of the hosting clause, cheap, and something nobody had done before signing. Rank: hosting and lock-in first, uptime during storm season second, latency third, cost last.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "rank lock-in first, cost last, because cost is the easiest one to fix once you notice it," and stop.
Cost: no legal budget to review every vendor contract line by line before launch. Say so honestly, and commit to reading just the termination and minimum-spend clauses first, not the whole document.
The model got better, for real: if the vendor drops its price next year, that's exactly when to re-check whether the same contract still lets you act on it, not assume a lower sticker price changes anything on its own.

Where people run it wrong.
They rank cost first because it's the easiest number to point at in a budget meeting.
They sign a multi-year contract at launch volume without modeling what the same terms mean at ten times the volume.
They treat "non-quality factors" as a single bucket instead of ranking them, so nothing gets checked before the contract is already signed.

How to use it live. The moment an interviewer asks for non-quality factors, ask yourself which one you'd regret not checking a year from now, not which one is easiest to name. Say that one first, and the rest of the list falls into a real order behind it.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0