Explain why the same prompt can cost different amounts on different days.

Halide turns a seller's flat product photos into a rotating, studio quality photo set called Turntable. Piran Tremaine owns the feature end to end. This is the week Moray Underdown, Halide's finance lead, nearly paused Turntable over a cost swing that had nothing wrong with it at all, and everything wrong with a dashboard that only ever showed one number.

How to answer this, stage by stage

Nobody is grading whether you can define a token. They're grading whether you can stop a false alarm before it becomes a Slack message that gets sent.

Let's learn

Here's what happens when the exact same button costs three times as much on a Wednesday as it did on the Tuesday right before it, with nobody touching a single line of code.

Halide turns a seller's flat product photos into a rotating, studio quality photo set, a feature it calls Turntable. Upload three to six photos from a phone, and Turntable hands back eight clean angles on a matching background, ready to drop into a listing. Before it, a seller either paid a real photographer a few hundred dollars a session, or listed with phone photos that looked exactly like what they were.

For months, Turntable's cost held still enough that nobody watched it closely. That's exactly why nobody caught the swing coming.

Turntable normally cost about eleven cents a render. That number held steady for months, somewhere between ten and thirteen cents, which is exactly why nobody watched it closely. Then, on one Wednesday, it jumped to thirty one cents. Two days later, on Friday, it jumped again, to twenty nine cents. Same code both days. Same prompt Turntable always sends. Nobody on the engineering team had shipped anything.

Here is the turn. The extra cost on Wednesday and the extra cost on Friday were never the same problem wearing the same number. One was real, ordinary usage running hot. The other was the provider quietly changing what it charged for the exact same job.

What it costs at its worst: on Wednesday afternoon, the finance lead had a message half typed, ready to pause Turntable for every seller uploading in bulk, on the theory that something was broken or being gamed. Pausing a live feature over a false alarm would have blocked real sellers mid catalog upload, for a swing that had nothing to do with a bug at all.

What I would leave alone: small sellers on a default backdrop, which is most of them. Their photos stay in a narrow size band, they almost never trigger a retry, and they always hit the backdrop cache. Adding a detailed breakdown to their view would just be noise for something that never moves.

The lesson: a prompt template can stay exactly the same in the code, forever. That was never a promise a render would cost the same every day, because the model only ever gets billed for what actually got sent to it, and that varies with whatever the seller sent Halide first.

Now here is the same thing as a story

Say the short version out loud in an interview. Read this one when you want to feel exactly how two spikes wearing the same shape of number can turn out to be two completely different afternoons.

Every Monday morning, before he opens anything else, Moray Underdown pulls up one spreadsheet tab and glances at one number: what it cost Halide, on average, to turn one seller's photos into a Turntable render the week before. He has done this since his first week at the company, three years ago, and for almost all of that time the number has done exactly one thing. Sat still, somewhere between ten and thirteen cents.

Piran Tremaine built Turntable's cost model herself, back when Halide had a dozen sellers and every one of them uploaded roughly the same kind of photo: a few phone shots, a plain background, nothing fancy. She priced it once, checked it every few months out of habit, and mostly stopped thinking about it. The number simply behaved.

Nine straight days of a clean deploy log, then two spikes two days apart. Nothing shipped either time.

Then came a Wednesday in April. By early afternoon, the day's running average had already climbed past twenty five cents, nearly triple. Moray watched it move for an hour, checked that nothing on Halide's pricing page had changed, and started drafting a message. He had it half typed by four forty: pause Turntable for any seller uploading in bulk until someone can confirm this isn't a bug, or someone gaming the free trial credits. He had his cursor on send.

Piran caught it first, over Slack, at four fifty two. "Give me an hour before you post that," she wrote. "I want to actually look at what rendered today before we tell people it's broken."

She pulled the deploy log first, the boring check. Nothing had shipped, not to Turntable's code, not to its prompt template, all week. She checked the billing line items against the provider's own invoice, cent for cent, to rule out Halide's own tracker double counting something. Both came back clean. Whatever this was, it wasn't a bug in the usual sense.

The words in the code stayed fixed. What those words were pointed at, every single render, never did.

So she recut the day by seller instead of trusting the average. Seventy percent of Wednesday's render volume, it turned out, had come from one account: Underbrush Outdoor, a camping gear seller, mid way through uploading its entire fall catalog. Three hundred and forty products, a lot of them reflective metal, kettles, folding stoves, tent poles, the kind of surface that catches light unevenly from every one of Turntable's eight angles. And for the first time, Underbrush had uploaded its own branded backdrop instead of using one of Halide's defaults.

Two things, once she actually measured them, explained almost the whole jump. Underbrush's photos were coming in around three times the usual pixel size, more detail in every single image Turntable had to work with. And the reflective gear kept failing Turntable's own lighting check, so it kept retrying: three point four passes per product on average, against the usual one point two. Add a fresh, uncached backdrop on top of that, and Wednesday's spike had a name.

Piran told Moray by five thirty. He didn't send the message. Underbrush's catalog finished uploading Thursday night, and Thursday's number was back to twelve cents before anyone else had even noticed there had been a Wednesday.

Then it happened again on Friday. Twenty nine cents.

One dollar figure was really four different things added together. Wednesday and Friday moved different parts of it.

This time nothing about the sellers looked unusual. Normal mix, normal photo sizes, normal backdrops, all defaults. Piran ran the exact same recut and it told her nothing, because there was nothing to recut. This time the average and every seller inside it agreed with each other. So she went looking somewhere she hadn't the first time: the provider's own status page. There it was, a capacity notice logged that afternoon, overflow traffic quietly routed to a higher priced serving tier for a few hours while the primary queue cleared. Nobody at Halide had been told. Nobody outside the provider's own engineers had any reason to know.

The decision Piran would take back sat in a planning meeting eighteen months earlier, the one where the team built Halide's very first cost dashboard. Twelve sellers, all roughly the same size, all on default backdrops. One blended number felt like plenty of detail for a business that small. Nobody argued. It was the right call for the company Halide was that day.

Same swing, same Wednesday. A dashboard that remembers the breakdown answers the question a single number never could.

Run the same Wednesday again, with the dashboard Piran built afterward: a breakdown by seller, average passes, average image size, cache hit rate, and which provider tier actually served the call. Moray still sees the number jump within the same five minutes it always has. But now the same screen answers the only question that matters, big honest upload or quiet price change, before he's finished his coffee.

What I'd tell myself, in that first planning meeting: a single number is only ever as honest as the business staying the size it was the day you built it. Halide didn't stay that size. The dashboard did, until Wednesday made it obvious.

TRACE, before you blame a bug that isn't there

Not a way to prove the feature was innocent both times. TRACE is what forces you to check usage before you trust a dollar figure, because the two spikes needed completely different fixes.

Two suspects explained Wednesday. A third, unrelated suspect explained Friday. The fourth never held up either day.

Three things worth saying plainly, since interviewers push here. Piran considered a second fix before the one she shipped: cap every product at one automatic generation pass, so cost per render would never move no matter what got uploaded. She rejected it, because a flat cap would quietly ship worse looking renders on exactly the products that need the retries most, the reflective and glossy gear where getting the lighting right is the whole reason a seller pays for Turntable at all. The AI specific failure worth naming by name is silent provider side routing: a vendor moves overflow traffic to a different, pricier serving tier during a capacity event, with no visible signal on the product's side at all. The guardrail is unglamorous: log which tier actually served each render, not just what it cost, and alert on a tier change, not only a price change. And the trade off, accepted on purpose, is real: keeping the retry budget flexible means accepting a cost per render that can triple on a hard day, in exchange for renders that still look right on the products where it matters most.

And if you want to be sure it really works, try it somewhere else

Same five letters, a packaged food label instead of a product photo, and this time one spike comes from a genuinely hard ingredient list, and the other from a change nobody at the company ever saw happen.

Victual drafts nutrition facts panels and plain language ingredient blurbs for small packaged food brands, checked against a compliance ruleset before a person signs off. Njoki Ingersoll runs product there, and hit a version of Piran's exact swing four months into the job, on a week when two label drafting spikes landed three days apart.

Different product, same shape of pipeline: a fixed prompt, a real submission filling it in, a compliance check that can send it back for another pass.

The first spike traced to one brand's spice blend, forty two ingredients long, several with names that sat right on the edge of Victual's allergen phrasing rules. The draft kept failing its own compliance check and re running, five passes before it cleared, against Victual's usual one. The second spike, three days later, hit a completely ordinary batch of single ingredient snack labels, nothing long, nothing unusual, and the average passes stayed normal. Njoki checked the model provider's status page and found the same story Piran had: a temporary switch to a higher priced serving tier during a capacity event, reversed by evening.

Mapped onto TRACE: the timeline shows two spikes, three days apart, on a clean deploy log both times. The recut splits cleanly, one brand's unusually hard submission against a completely ordinary batch that had nothing to do with the spike at all. The assumption Njoki ruled out first, the same one Piran ruled out, was a billing bug: the invoice matched the internal tracker exactly. The cause candidates were the same shape, ingredient count, compliance retries, cache reuse on repeated boilerplate phrasing, and provider tier. The evidence test gave the same answer: pass counts explained the first spike and stayed flat through the second, so the second had to be a price the provider changed, not work Victual asked for.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: same prompt template, different filled in prompt, check usage before you trust the dollar figure.
Cost: no time to trace a real incident. Ask one question instead: on the expensive day, did the actual work done, passes and content size, move by the same amount as the bill?
The model got better, for real: say the provider drops its price twenty percent next quarter. Usage driven spikes, big uploads and hard retries, don't shrink at all, because they were never about price in the first place.

Where people run it wrong.
They see "same prompt" in the code and assume "same bill," instead of checking what actually filled the template that day.
They trust the dollar total on its own and skip straight to "something's broken," instead of checking usage first.
They fix a price spike by capping retries, which quietly breaks quality on hard cases instead of fixing the actual cause.

How to use it live. When an interviewer asks this cold, buy two seconds by asking one thing back: "are we talking about the same prompt template, or the same fully filled in call?" That question alone is usually exactly what a question shaped like this one is listening for.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0