Defend a model choice to an engineer who prefers a different one.

Duskfarrow Systems builds PatchWright, a coding copilot that drafts a code change from a plain-language request and hands it to an engineer to review. Desmond Achterberg is the AI PM who picked the model PatchWright runs on. Piotr Zielinski is a senior engineer on the platform team who thinks a bigger, more famous model would serve the product better.

How to answer this, stage by stage

Nobody is scoring whether your model choice is objectively correct. They're scoring whether you can turn "I disagree" into a number both of you can check.

Let's learn

Say we build a tool that writes a first draft of a code change from a plain-language request, and hands it to an engineer to review before it merges.

Before PatchWright, an engineer wrote every function by hand, about 20 minutes for a typical small change. With PatchWright, a draft comes back in under ten seconds, and an engineer reviews it instead of writing it from scratch.

The five letters, held up as one page. Own the numbers is the one most model defenses skip.

Here's the turn: the model Piotr prefers scores 91 percent on a public coding benchmark, 8 points higher than the model Desmond chose. On Duskfarrow's own 300-PR golden set, checked against their real internal conventions, it only produces a diff that passes without a rewrite 74 percent of the time. Desmond's choice, lower on the public score, passes 92 percent of the time on that same real test.

At its worst, engineers spend more hours fixing a "smarter" model's suggestions than they would have spent writing the code themselves, while the monthly bill still looks small because it only ever counted the tokens.

What I would leave alone: I wouldn't build this same cost model for a low-stakes internal script generator with no real users, since a failed suggestion there costs a shrug, not an engineer's afternoon.

The lesson: a preference defended with adjectives loses to a preference defended with numbers, but only if somebody actually builds the numbers first.

Now here is the same thing as a story

The short version above is what you'd say in the room with Piotr. Read this one for what it felt like the eight months before anyone built the numbers at all.

For eight months, Desmond's model choice was uncontested. Then Piotr got a look at the benchmark.

Desmond had picked PatchWright's model after running it against a handful of sample requests during evaluation, and it had shipped clean: engineers liked the speed, and nobody complained. For months, code review caught the occasional odd suggestion and moved on, the way it always had.

The fourth step is where a public benchmark's whole promise either holds up or quietly doesn't.

Piotr, newer to the platform team, pulled up a public leaderboard during a planning meeting and pointed out that a bigger, more expensive model scored 8 points higher. "You picked the worse model because it's cheaper, not because it's better," he said, not unkindly, just plainly, in front of the team.

Desmond didn't have a ready number. He said something about "it felt more reliable in our testing," which was true and also nothing anyone could check. Piotr, unconvinced, quietly set up a personal routing wrapper that sent his own team's PatchWright requests to his preferred model instead, behind a feature flag nobody else knew existed.

Same word, two different questions, and only one of them was ever actually tested.

Two months later, a bug pattern started showing up only in PRs from Piotr's team, a specific internal error-handling convention getting silently ignored in generated code. Nobody connected it to the model until someone noticed his team's diffs looked stylistically different from everyone else's.

The wrapper wasn't dishonesty. It was the only move available with nowhere shared to actually check.

The real question was never whether Piotr's preferred model was more capable in general. It was whether Duskfarrow had ever built a way to check, on their own real work, which model actually cost less once a failed suggestion needed a human hour to fix.

Four parts, and the argument in that planning meeting had only ever touched one of them, price per token.

When PatchWright's model was first picked, someone said, "let's just go with what tested well and move fast," and it sounded reasonable, since at the time, nobody had reason to think a second opinion was coming.

Rerun the same planning meeting with the golden set and the cost equation already built: Desmond states the equation, owns the 3,000-token and 40,000-request assumptions from real usage logs, gives a range on rewrite time, and shows the $124,560-a-month gap before Piotr ever reaches for the leaderboard. The shadow wrapper never gets built, because there's already a shared place to check.

What I'd tell myself, hearing "you picked the worse model" land in that meeting: a preference with no number behind it isn't a defense, it's an invitation for someone else to build their own workaround instead.

BOUND, the defense that turns a disagreement into a numberNot a script for winning every argument with math. BOUND is what tells you exactly which number the disagreement is actually about.

The recap, one line per letter: break it down is tokens times price plus failures times rewrite time, own the numbers is stating where the 3,000 tokens and 12 minutes came from, use a range is the $120k to $240k gap instead of one number, nail the sanity check is explaining the huge total before anyone else questions it, and direction is naming rewrite time as the assumption most worth watching.

And if you want to be sure it really works, try it somewhere elseSame five letters, a textile mill instead of a codebase. The engineer disagreeing changes, the missing golden set doesn't.

Ingeborg Vasquez runs product at Cascade Mills, where ThreadScope flags likely fabric defects on the line for a quality inspector to confirm. A vendor's newer vision model scores higher on a public fabric-defect benchmark, and a process engineer wants to switch immediately. Mapped onto BOUND: break it down is cost per bolt of fabric equals inspection time saved minus the cost of a missed defect that reaches a customer. Own the numbers means pulling Cascade's own 4,000 tagged photos from their specific looms, not the vendor's demo set. Use a range covers the cost of a missed defect, which varies by fabric type. Nail the sanity check means explaining why the vendor's higher public score doesn't show up as fewer missed defects on Cascade's own looms. Direction names the one thing that would flip the pick: if the vendor's model gets tuned on Cascade's own defect photos.

Different factory floor entirely, same missing half of the comparison.

Four steps, and skipping straight to the last one is exactly what invites a shadow workaround instead.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "defend it with the cost of a failed suggestion on our own work, not a public score," and stop.
Cost: no time to build a full golden set before the meeting. Say so honestly, and bring even 30 real examples instead of zero, rather than defending with adjectives alone.
The preferred model really is better, for real: if the numbers actually favor Piotr's pick once tested on the golden set, that's the answer, and saying so out loud is what makes the whole method trustworthy instead of a way to always win.

Where people run it wrong.
They defend a model choice with confidence instead of a number the other person can check themselves.
They quote a public benchmark score as if it settles a question it was never built to test.
They let a disagreement fester with no shared place to check it, until someone builds a private workaround instead.

How to use it live. The moment an engineer prefers a different model, ask yourself: what does a wrong suggestion actually cost, in hours and dollars, on our own real work? Build that number, own where it comes from, and the rest of the defense follows on its own.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0