How do you decide whether to disable a feature or degrade it during an incident?

How to answer this, stage by stage

Nobody is grading whether you know the word "graceful degradation." They're grading whether you can commit to a side and show why the two kinds of wrong don't cost the same, even when the invoice says otherwise. Seven moves get you there.

Let's learn

What happens the first time an AI travel assistant is confidently wrong about a room that's already gone?

Say Farsight Travel builds Roadmark, an assistant that drafts a full trip, flights, a hotel, and a short list of things to do, in one pass. It pulls its prices and its availability straight from the real booking systems behind Farsight, not from a guess.

Before Roadmark, a Farsight trip planner spent about forty five minutes stitching a five day trip together by hand, checking one hotel and one flight at a time. Roadmark builds the same trip in under twenty seconds, prices and all. Now, about eighty five thousand people a day ask Roadmark to plan something. Most days it just works.

Then a hotel partner's live price feed goes down. The extra downtime is not the problem by itself. The real problem shows up only if nobody built a hard stop for that moment, and the assistant is instead allowed to keep going on the last prices it saw, dressed up exactly like a fresh one.

Two very different kinds of wrong, drawn side by side. One is loud and cheap. One is quiet and expensive.

At its worst, Roadmark keeps drafting full, confident trips through the whole outage. A family pays for a room that sold out thirty eight minutes earlier. They find out at check-in, in another country, with nowhere to sleep. That's worse than if Farsight had never built Roadmark at all, because a person planning by hand never gets confidently told a sold-out room is theirs.

The choice I would take back. When Roadmark was first built, the plan for a partner outage was a small banner at the top of the screen: "prices may not be fully current." That felt like enough. It also meant the only thing standing between a customer and a booking on a stale price was whether they happened to read one line of grey text above a trip they were excited about.

What I would leave alone. Roadmark also ranks which restaurants and activities to suggest each day, based on what similar travellers liked. That part never touches a real price or a real booking. During the same outage, it kept running exactly as it always does, a little less personal for a few minutes while some data refreshed, and nobody's trip was ever at risk because of it.

The lesson. A warning banner is not a safety check. It's a hope that someone reads the fine print at the exact moment they're excited about a trip. If what matters is whether a person can actually book what they're looking at, that has to be checked again, by the system, right before they pay, not left to a sentence they can scroll past.

Now here is the same thing as a story

The short version is above. Read on if you want to feel how close that Kyoto trip came to actually going through.

Fabienne Villareal has run the Roadmark team at Farsight Travel for three years. Before that she spent four years in customer support, which is why she still reads every complaint with the word "refund" in it, even the ones nobody escalates to her.

For most of a year, the small grey banner felt like enough. Partner outages were rare, usually gone in ten minutes, and nobody had ever complained about a stale price making it all the way to checkout.

Then, on a Thursday evening, a hotel partner's feed went down for forty five minutes. A family building a nine day trip to Japan asked Roadmark for a boutique inn in Kyoto. Roadmark found one, a small place near the river, and built the price straight into their trip, the way it always does.

That inn had sold out thirty eight minutes earlier.

The family didn't know that. Neither did Roadmark. They reviewed the trip, added the inn to their cart, and moved to pay. The grey banner had been sitting at the top of the screen the whole time. Nobody in the family remembers reading it.

Fabienne found out the next morning, from a support ticket with the subject line "card declined, room says available." She pulled the trip, checked the timestamps, and sat with it longer than the ticket needed.

She asked the data team one question: over the last month, across every partner outage, how many trips had made it that far, all the way to the payment screen, on a price or a room that was already gone.

The answer came back the next day. Three hundred and forty. Eleven of them didn't get the same lucky break as the Kyoto family. Eleven customers had a booking fail at the very last step, after they thought it was done.

The old decision, told plainly. When Roadmark was first designed, the team talked about building a hard check right before payment, one more call to the real booking system, at the exact moment someone tries to pay. It got cut. It added a few seconds to checkout, and outages were rare enough that nobody thought it would matter.

Farsight paid the eleven customers a hundred and fifty dollars each in goodwill credit, about sixteen hundred and fifty dollars total. That part was easy to write down. What didn't fit on the same line was the forty hours the support team spent working the tickets, the two customers who posted about it publicly, and the six point drop in Roadmark's own trust score the following week, across everyone who used it that week, not just the eleven.

The replay came two months later. A different hotel partner's feed went down, this time for fifty minutes. This time, the moment the feed started returning old timestamps, Roadmark's price and availability layer shut itself off, cleanly, for that partner, while trip planning and activity ranking kept running. About two thousand nine hundred and fifty sessions hit that quiet screen. Farsight's own numbers, pulled afterward, showed seventy four percent of those people came back within a day and finished the trip anyway, once the feed was back or by booking it themselves. The net loss was about seven hundred and sixty seven sessions, an estimated twenty eight thousand dollars in trip value that never got rebooked that week.

Twenty eight thousand dollars is a real number and it stings to write down. But it's the whole cost. Nobody found out at check-in that their room didn't exist. Nobody got a hundred and fifty dollar apology. Nobody posted about it.

What Fabienne would tell herself, back when the hard payment check got cut to save a few seconds: a few seconds is a real cost. So is finding out, months later, how many families almost paid for a room that wasn't there, and knowing eleven of them didn't get the lucky break the one you heard about did.

The four letters behind "turn it off"

This is a straight A or B question, disable or degrade, so PICK fits, not a framework built for a habit that snaps or a number that needs sizing.

P, position. Say the pick before the reasoning. Disable the part of Roadmark that could be confidently wrong about something bookable, price, room, or date, the moment an incident touches it. Keep everything else running.
I, impact. A disabled screen costs a customer a few minutes, absorbed and mostly forgotten by the next day. A quietly wrong screen costs a customer a booking they thought was real, discovered at the worst possible moment, at payment or at check-in.
C, cost asymmetry. The dollar cost of the quiet-wrong path, about thirty four hundred fifty dollars in credits and support hours, looks smaller than the dollar cost of disabling, about twenty eight thousand dollars in unrecovered bookings. The real cost isn't on that line. It's the six point trust drop across everyone who used Roadmark that week, not just the eleven people it happened to, and that number never shows up on a single incident's invoice.
K, kill criteria. Flip the pick when the incident doesn't touch price, room, or date truth at all, only ranking or personalisation, or when a real, live check sits between the AI's answer and the moment someone commits to it. Both were true for the personalisation layer during the outage. Neither was true for the price feed, so it came down.

• P: disable the part that could be confidently wrong about something bookable.
• I: minutes lost against a booking lost, felt by the same customer, at very different moments.
• C: the cheap number is the one that gets written down. The expensive one is the one that doesn't.
• K: flip it only when the booking truth isn't at stake, or a live check catches it before it becomes real.

And if you want to be sure it really works, try it somewhere else

Bramblecroft Veterinary Group runs twelve clinics and an intake assistant called SortLine. A vet tech enters an animal's vitals and the owner's symptoms, and SortLine scores how urgent the case is in about four seconds, instead of the six to eight minutes it used to take a tech to work the same triage math by hand.

P, position. The moment the vitals sensor gateway starts sending stale or missing readings without erroring, SortLine's urgency score turns off. Techs fall back to the paper checklist they already know.
I, impact. Disabling costs a tech about four extra minutes per animal, absorbed into a normal busy afternoon. Letting SortLine keep scoring on old vitals risks ranking a genuinely urgent animal, one with bloat or trouble breathing, as routine, and nobody notices until it's worse.
C, cost asymmetry. Four minutes, many times a day, is a real cost, but it's a cost every clinic already knows how to absorb. One missed urgent case is an animal's life and a story that spreads through a small town faster than any goodwill credit can catch up to.
K, kill criteria. If the outage only touches something that never feeds the urgency score, an owner's past visit history, say, SortLine's score keeps running, and only that unrelated part degrades.

Swap the trigger and it still runs.
Speed: an interviewer gives you ninety seconds. Skip straight to the pick, turn off whatever the AI could be confidently wrong about, keep the rest running, check the truth again right before it becomes real.
Cost: engineering says the payment-time check adds real delay at busy season. Don't cut it to save the time, that's the whole safety check; accept the extra second instead, and say so out loud as the cost you're choosing to pay.
The model got better: Roadmark's price-matching model turns out to have handled that particular outage fine, verified after the fact. That doesn't make disabling wrong. Nobody could have known that live, with what was actually known at the time.

Where people run it wrong.
They treat "the screen still answers" as proof it's fine, when answering and being true are two different claims for something that generates its answers instead of looking them up.
They set the safe-to-degrade line as a feeling, "seems to have settled down," instead of a number checked live.
They fix a stale-data problem by swapping in a faster or cheaper model, which does nothing, because the data feeding it was never the model's fault.

How to use it live. Say the pick out loud in the first sentence, before any reasoning: "my default is disable, here's why." That buys you the rest of the answer to unpack the asymmetry, instead of spending your first thirty seconds sounding like you're still deciding.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0