Explain how to structure pricing so cost increases do not require a repricing.

A tool that reviews every pull request can price itself once and forget about it, right up until the model has to get better and the cost behind that number stops sitting still. This is about what to build into the price on day one, so a rising cost never turns into an email nobody wants to send.

How to answer this, stage by stage

Nobody is grading whether you can name a pricing model, usage-based, seat-based, tiered. They're grading whether you know that inference cost moves on its own, and whether you built anything that absorbs that move without a customer ever seeing it.

Let's learn

Here is what happens when a tool's own cost of thinking creeps up quietly, long after the price was ever set.

Patchlight is a tool that reads every pull request a team opens and flags bugs, security issues, and style problems before a person reviews it.

Before Patchlight, two engineers read every diff by hand, about thirty five minutes each, for pull requests landing roughly ninety times a week.

With Patchlight, that same pull request gets a first pass in under a minute. Norstead Systems signed on sixteen months ago: four thousand review credits a month, thirty cents each, one credit per pull request, no matter how big the diff or which model actually read it. Behind that flat number, Patchlight's own cost to run the review sat at about four cents, a small, cheap model reading diffs that averaged ninety lines changed. Eighty seven cents of margin on the dollar, and nobody at Norstead ever asked what a credit cost to deliver, because the bill matched the invoice every single month.

One flat number on Norstead's side. A moving number on Patchlight's side. Nothing in the price connected the two.

Norstead's codebase grew into a single monorepo over those sixteen months, and pull requests grew with it. The average diff size climbed from ninety lines changed to three hundred and ten, over five quarters, as more of the company's code lived in one place and touched more of it at once.

Here's the turn. A pull request three times the size is not, by itself, a crisis. The real problem is what happens next, when nobody has built anything to absorb that rise. Someone reaches for the fastest lever within reach: cap the expensive model, quietly, and let the cheap one take whatever's left over.

At its worst, that quiet cap did real damage. On the larger diffs Norstead was now shipping, the cheap model's own catch rate for a planted auth bug, checked against Patchlight's eval set, had already slipped from eighty nine percent down to fifty eight percent, just from the size of the diff. Capping the strong model on top of that meant the exact pull requests most likely to hide something serious were the ones most likely to get the weaker read.

Four things sitting behind one flat number. The customer only ever sees the number.

What I would leave alone: the thirty cent sticker price itself, and the one credit, one pull request abstraction. That part is good and worth keeping at any size. Small pilot accounts under a few hundred pull requests a month don't need the full risk classifier either, running everything on the strong model there is cheap enough in absolute dollars that the extra machinery would only slow a champion down for a threat that hasn't shown up yet.

The lesson: a pricing conversation about cost was never really about the price. It's about whether the routing behind that price was ever built to answer to anything but the price itself.

Now here is the same thing as a story

Read the short version above when you're in the room. Read this one when you want to feel why fourteen thousand dollars moved before anyone but a fraud team noticed.

Deniz Sokolowski built Patchlight's pricing the way she built everything: with the invoice already imagined before the feature shipped. Two years running pricing and packaging, and she'd never once had to explain a bill to a confused customer. The math was clean on purpose. One credit, one pull request, thirty cents, whatever ran behind the screen was Patchlight's business, not the customer's.

For most of a year, that cleanliness held. Norstead Systems signed on early, four thousand credits a month, and Arvo Vandermolen, their VP of Engineering, never once emailed to ask what a credit actually cost to deliver. He didn't need to. The bill matched the invoice every month, and Patchlight's own margin sat quietly in a spreadsheet nobody outside the company read.

Eight months of a two minute habit, then two quiet handoffs, before the cap that mattered ever reached her desk.

It thinned in three beats, same as anything that erodes instead of breaking. In the first eight months of Norstead's monorepo migration, Deniz personally read every change to the routing configuration before it deployed, a two minute habit, checking that payment and auth paths still forced the strong model no matter what. By month nine, with the monorepo three times its original size and pull requests growing to match, she'd handed that review to whoever was on call that week, just to move faster. By month thirteen, when an engineer added a monthly cap of a hundred and fifty premium reviews per account to keep a ballooning infrastructure bill in check, nobody brought the change back to her at all. On paper it looked like an infrastructure fix. It was a pricing decision wearing an infrastructure fix's clothes.

The trigger wasn't the size of the bug. It was one message. Arvo forwarded a ticket from Norstead's own fraud team, eleven days after a pull request called #3,912 had merged, with one line typed above it: "Why didn't Patchlight catch this."

Pull request #3,912 refactored the signature check on Norstead's payment webhook handler. It landed on day twenty three of the billing cycle, after Norstead had already used all one hundred fifty of its premium reviews that month. So it fell to the cheap tier, the way anything did once the cap was hit, no matter what it touched. The cheap model missed a broken check that let a replayed webhook call slip through twice. Duplicate payouts went out on the same transaction. By the time the fraud team traced it, about fourteen thousand dollars had already moved, and it took roughly thirty hours of cleanup to unwind it.

Same pull request, same bug hiding in it. Only the reason it got the cheap model changed.

Fifteen months earlier, in a meeting that ran fifteen minutes, Patchlight's two person go-to-market team decided the product would launch with a single model tier, one setting, for every pull request. Somebody asked whether they should build the risk-based routing in from day one. The answer was no. It was the reasonable call. The company's biggest account at the time sent maybe forty pull requests a week, none of them over a hundred lines.

Run the same day again, with the risk classifier live from the start. Pull request #3,912 gets flagged high risk the moment it's opened, because it touches a payment webhook path, not because of anything about the calendar or the month's budget. It routes to the strong model automatically, cap or no cap, and the broken signature check gets flagged in the same review pass, in under two minutes. Norstead's blended cost for that account that month lands around nineteen cents a credit, margin down to thirty seven percent from the eighty seven percent it started at, but still healthy, still inside the buffer Patchlight built for exactly this. Nobody has to write an email about a new price.

One design let a shrinking monthly budget decide which model read the code. The other let the code decide, and let the budget absorb whatever that decision cost.

What I'd tell myself, back in that fifteen minute meeting: the one-model decision wasn't wrong for the product that existed then. It was wrong for the product everyone already knew Norstead's engineers were going to grow into, one merge at a time. Ask how big the diffs get before deciding the routing can wait.

SPARK, or the five decisions behind one credit that never has to move

Not a list of pricing models to pick between. SPARK names the one design decision a rising cost has to run into before it ever reaches a customer, and asks what breaks the day that decision is wrong.

Five letters. The anchor is the one the whole answer hangs on.

One branch shipped. Three stayed on the board on purpose.

Three things worth stating directly, since this is where the real judgment sits. The alternative Patchlight's team considered, and rejected, was forcing every pull request onto the strongest model the moment the average diff size started climbing, uniform quality, no routing logic to build. It lost because at Norstead's real volume that blends out to forty two cents a review against a thirty cent price, a design that goes underwater the day it ships, not months later. The AI-specific failure worth naming by name is silent quality drift under cost pressure: a model tier gets swapped or capped for reasons that have nothing to do with what the code actually needs, and nobody notices until a bug that should have been caught gets through. The guardrail is the risk classifier itself, reading the changed file paths before the router ever checks the account's remaining budget, so a payment or auth change can never be assigned by anything but its own risk. And the trade-off is real: routing by risk instead of by cost means Patchlight sometimes runs the expensive model on a pull request that turns out to be harmless, eating margin on purpose, in exchange for never once explaining to a customer why the model missed something it was supposed to catch.

And if you want to be sure it really works, try it somewhere else

Same five letters, a radiology read instead of a pull request, and this time the thing that grows isn't diff size. It's image resolution.

Sigrun Oakvane runs imaging operations at Greywick Diagnostic Imaging. Clarolens reads chest and limb X-rays for anomalies, one flat sixty cent read credit, same price no matter which scanner took the image. As Greywick upgraded its scanners from two megapixels to fourteen over eighteen months, the cost to run Clarolens's vision model on each image climbed right along with the resolution.

The rejected alternative here: a resolution-tiered price, charging more for a higher-resolution image. It was on the table and it lost, because it's a repricing event wearing a different name, the number a clinic pays would still move every time a scanner gets upgraded. Instead, Clarolens routes by clinical urgency tag at intake, chest pain and stroke-code scans always get the top model tier, routine follow-ups get the cheap one, and the sixty cent price never has to know which scanner took the picture.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: route by what the pull request touches, not by what's left in the budget, full stop.
Cost: there's no engineering budget this quarter for a real risk classifier. Ship the cheap version first, a manual list of protected paths, payment, auth, migrations, checked by a simple rule instead of a trained model.
The model got better, for real: say Patchlight's cheap tier doubles in accuracy overnight. The fix barely changes. A well-built router still decides by risk, it just gets to run more pull requests through the cheaper tier without giving anything up.

Where people run it wrong.
They let the price track the underlying cost directly, a resolution surcharge, a per-token line item, and end up with the very unpredictability the flat unit was supposed to remove.
They watch blended margin only, and miss that the high-risk slice of their volume has already gone underwater while the average still looks fine.
They fix a margin problem by adding a cap, which protects the number this month and quietly breaks quality on exactly the inputs that mattered most.

How to use it live. Say the split out loud before answering: "is the cost actually rising because the model changed, or because what customers are sending it changed, because those need two different fixes." That buys a beat, and shows the interviewer you know a model upgrade and a shifting input distribution aren't the same problem.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0