Describe a case where adding AI increased user effort rather than reducing it.

Larkstone Outfitters sells camping and hiking gear through live chat support. Cerelia Culloway has worked the damaged item and refund dispute queue for four years. Once Larkstone's drafting tool started writing her replies for her, she built on top of its suggestions like everyone else, until one Thursday she caught a wrong number seconds before sending it, and never let the tool's numbers near a reply again.

How to answer this, stage by stage

Nobody is grading whether you can describe a tool that backfired. They are grading whether you'll notice that "we generated a draft fast" quietly got treated as "we made the job faster," when nobody ever measured the second one.

Let's learn

Every damaged item dispute used to take Cerelia about nine minutes, start to finish, before Larkstone Outfitters ever put a model near her queue.

Larkstone sells tents, stoves, and hiking packs online, and its whole support team works over live chat, about 1,400 conversations a day. Around 95 of those, every day, are damaged item and partial refund disputes: an item came back scratched, or dented, or the wrong size, and someone has to work out what's owed. Before the drafting tool, an agent opened the order record, checked the item's history, worked out the refund amount and whether a damage credit applied, then wrote the reply herself. Cerelia was good at it. She could spot a mismatched SKU before the page finished loading.

The drafting tool reads a new ticket and writes a full reply: an apology, the refund amount, the eligibility line, and a sign off, in one smooth paragraph. Larkstone's launch review measured two things before it shipped: time to first draft, under 2 seconds, and how much of a sent reply changed from what the model wrote. On damaged item tickets in the first month, 81 percent of drafts went out with fewer than 15 words changed. That looked like a win worth telling leadership about.

Nothing about the ticket changed. What Cerelia required before trusting a number changed completely.

Here's the turn. The draft's overall accuracy on this ticket type never was the problem. It got the refund amount or the eligibility line wrong about 1 in 6 times, and that number barely moved the whole time. The real problem is what Cerelia had to do once she knew that. She couldn't just check a little more carefully. The moment she stopped trusting one number in a drafted reply, she couldn't trust any of them without checking, so for this ticket type, she started opening the order record and working out the refund herself, every single time, before she'd let herself read the draft at all.

At its worst, that gap isn't just a story about one agent. Multiply Cerelia's extra two and a half minutes across roughly 95 damaged item disputes a day, across a queue of agents doing the same ticket type the same way, and this one ticket type alone was costing Larkstone's support floor about four hours of agent time a day that didn't exist before the tool arrived. Worse than never building it, because the nine minute baseline was already fine.

One box in the middle did two jobs at once: state the fact, and phrase the sentence. Nothing marked where one ended and the other began.

What I would leave alone: shipping status and delivery date questions on this same live chat tool don't need any of this. The draft states a date pulled straight from the tracking number, the same fact every time, nothing generated. No agent on that ticket type ever built a habit of rebuilding it by hand, because there was never a sentence hiding a guess inside it.

The lesson: a fast draft and a fast resolution are not the same claim, and the only way to find out you've been treating them as one is to actually go measure the second thing. Larkstone had a whole month of numbers that looked like proof. None of them were measuring the thing that mattered.

Now here is the same thing as a story

The short version above is what you'd actually say in the room. Read this one when you want to feel exactly what 150 dollars, caught by luck, does to a person's whole approach to a tool.

Every shift, before the drafting tool, Cerelia opened the same second tab first: the order record. Check the item, check the return reason, check what the customer already paid, then write the reply. She'd worked the return and damage queue at Larkstone for four years, long enough that new hires got sent to her when a ticket didn't make sense. Ask her whether a scratched tent pole qualified for a damage credit or a straight refund, and she'd know before she finished reading the description.

The drafting tool arrived in the spring. For the first few months, on this ticket type, it was genuinely good. It read the ticket, pulled up what looked like the right numbers, and wrote a reply that sounded like something she'd have typed herself. She still opened the order record those first weeks, mostly out of habit, and it kept matching. So she opened it a little less. Then only for the bigger orders. Then, without ever deciding to, she stopped opening it at all for this ticket type, the same way she stopped for the easy shipping questions months earlier, because the draft kept being right and there was no reason left to check.

Nobody told her to stop checking. The draft simply kept being right long enough that checking stopped feeling like part of the job.

Then came a Thursday. A customer named Vasyl Brindley had returned a camp stove with a cracked burner ring, and the draft told Cerelia he was owed 340 dollars: the item's price, plus a 150 dollar damage credit. She had her cursor over send. Out of the same old habit that had mostly gone quiet, she glanced at the order total sitting in the sidebar. It said 190.

The damage credit only applies when a customer keeps a flawed item instead of returning it. Halden was returning his. The draft had pulled the credit rule from a similar ticket earlier that week and applied it here anyway, in a sentence that read just as confidently as every correct one she'd sent that morning.

Here's what she did next, and it's the whole story. She didn't start reading the draft's numbers more carefully. She stopped reading them at all. From that Thursday on, for every damaged item dispute, she opened the order record first, worked out the refund and the eligibility herself, wrote the number down, and only then read the draft, purely to borrow a line or two of its wording. She never told anyone this became her process. Nothing in Larkstone's tools tracked it. The dashboard still saw a draft generated fast, and a reply sent with only a few words changed, because her rewritten numbers usually landed close to what the draft had guessed anyway.

We considered the easy fix first: put a line under every drafted reply, please confirm the amount before sending. We killed it within a day. Cerelia already knew to check. That was exactly the problem, not the fix. A reminder doesn't tell you which of the draft's four sentences is the one that's wrong, so it doesn't save a single minute, it just makes the checking official.

We also considered training the model on more damaged item examples until the error rate came down. We killed that one too, more slowly, because it took a real conversation to see why. Even a model that's right 99 times out of 100 is still, on the hundredth, confidently wrong in a sentence that looks exactly like the other 99. Cerelia can't tell which sentence is the hundredth one by reading it. Neither can anyone. A lower error rate makes the mistake rarer. It doesn't make it visible.

Here's the decision I'd take back instead, and it isn't Cerelia's, it's the one made months earlier, in the room where the drafting tool got designed. Someone decided the reply should read as one smooth, human sounding paragraph, the refund amount and the eligibility line generated in the same breath as the apology and the sign off. Reasonable, at the time. The early version only handled simple, single item tickets, where the numbers were nearly always right, and a paragraph with no visible seams felt like a better product than a form letter with blanks in it. Nobody built a second lane for the day the ticket types got messier than that.

Run the same kind of ticket again, six weeks after the fix ships. A different camp stove, a different customer, the same kind of damage credit mix up the model would have made before. This time the refund amount and the eligibility line don't come from the model at all. They're pulled straight from the order system and shown as a marked field, right in the reply composer, next to the AI written sentence around them. The eligibility rule carries its own confidence line: under 90 percent sure, and it doesn't state a value, it flags the ticket for a person instead. Cerelia reads the field once, confirms it against the order total already sitting in the sidebar, and sends. No second tab. No rebuilt reply. Handle time for this exact kind of ticket: six and a half minutes, faster than the tool ever managed, faster than the nine minutes it took before the tool existed at all.

What I'd tell myself, back in that first design meeting: making a reply sound like one smooth paragraph and making a fact something you can trust are two different jobs. We built a tool that was very good at the first one, and quietly assumed that meant it was doing the second.

FLIPS, or the fact that stopped being a sentence

Not a trick to sound structured. It's the difference between a reply that sounds finished and a reply that's actually made of the right numbers.

Four setup and payoff letters, and one question that only mattered once a model was allowed to state a dollar figure on its own.

The whole answer, in one picture. The launch review designed for a dial. Cerelia lived inside a switch, for nine quiet weeks before anyone else noticed.

Two of these four pieces are facts now. The other two are still the model's job, and always were.

The AI specific failure worth naming plainly is confident wrongness: a model stating a dollar figure or a policy exception in fluent, certain sounding prose, with no visible difference between a right answer and a wrong one. The guardrail is the verified field itself, backed by a real confidence threshold on the eligibility classifier, tuned against six months of resolved disputes rather than a guess, so anything under 90 percent routes to a person instead of asserting a value. There's a real trade off accepted here too: building the live link into the order system took about five weeks of engineering time the team didn't plan for at launch, and the tool now sometimes shows "needs a policy check" instead of a tidy, fully finished looking paragraph, on purpose, trading a little polish and shipped speed for two facts an agent can actually trust without opening a second tab.

And if you want to be sure it really works, try it somewhere else

Same five letters, a genuinely different flip family this time. Nobody here stops trusting the tool. She just starts saving it for exactly the wrong cases.

Wrenwick Veterinary Partners runs a group of clinics, and its vet techs use a drafting tool to write post visit care instruction messages that go to pet owners: medication timing, follow up scheduling, what to watch for. Kirstyn Beckendorf is known on her team for the toughest kind of message to get right, post surgical cases on two or three overlapping medications with a tapering schedule. For the tool's first two quarters, she leaned on it for every visit type, routine and complex alike, because it read to her like a fast, reasonable first pass either way.

Same method, a different verb entirely. One agent stopped trusting the tool. The other started rationing it, straight toward the cases it's worst at.

The trap showed up after a cost review capped every tech at 200 AI assisted drafts a month, to control the clinic group's model spend. Kirstyn didn't abandon the tool. She started budgeting it. Routine wellness visit messages, the kind she can write from memory in under a minute anyway, she now writes herself and saves her credits for the complex, time consuming, post op cases, the ones she cares most about getting right and where writing from scratch takes the longest.

F · Kirstyn Beckendorf, a vet tech at Wrenwick who writes post visit care instructions, especially skilled with multi drug taper schedules.
L · She used to reach for the drafting tool on every visit type without thinking about which ones it was actually good at.
I · The substitution flip, a different shape entirely from Cerelia's. Old setting: she uses the tool for every message, easy and hard alike. New setting: with a fixed monthly quota, she rations her limited credits toward the visit types she cares most about getting right, the complex multi drug cases, which happen to be exactly the visit type the model is worst at. The model never changed. The mix of what it's being asked to draft did, and the measured error rate on AI assisted messages climbed because of it.
P · Whoever set the quota priced every visit type's AI credits the same, instead of steering the limited credits toward the visit types the model was actually reliable on.
S · Wrenwick reserves AI drafting automatically for visit types where the model's fact error rate sits under 5 percent, routine wellness and vaccine follow ups. For multi drug post op cases, techs get a structured template that pulls exact dosages straight from the medication record instead of full free text drafting. Post op message time, previously creeping toward 14 minutes as Kirstyn wrote them fully by hand to save her quota, drops to under 5.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: the model didn't get worse, the mix of what it was asked to draft got harder, and no dashboard watching an overall average would ever show you that on its own.
Cost: no budget for a full verified-field build like Larkstone's. Even a simple rule, flag any reply with a number over a set dollar threshold for a mandatory second look, catches most of the damage before a bigger fix ships.
The model got better, for real: say next quarter's accuracy climbs across the board. Doesn't matter, maybe it matters more. An improving average is exactly the kind of number that hides one ticket type still quietly getting worse underneath it.

Where people run it wrong.
They treat "drafts are going out nearly unedited" as proof the tool is working, instead of asking whether that's true on the hard cases too, or only on the easy ones nobody had to check anyway.
They fix trust with a reminder to double check, instead of asking which specific facts inside the reply are generated versus looked up.
They wait for a customer complaint to force the real comparison, instead of instrumenting total ticket time by type from the day the tool ships.

How to use it live. If an interviewer hands you a tool that's supposed to save someone time, ask yourself one thing before answering: which two or three facts in its output, if wrong, would actually cost someone money or trust, and are those facts generated by the model or looked up from somewhere real? That question alone usually is the whole answer.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0