List five conditions under which you should reject an AI solution outright.

How to actually say this in the room

Seven moves. This is a commit question wearing a list's clothes, so say the position before you count anything on your fingers.

One more thing before the walkthrough moves on: these five conditions apply to a piece of the product, not the whole thing. Most candidates hear "list conditions to reject AI" and answer as if the question were "when do you turn the feature off." Say which part of the workflow each condition actually touches, and you've answered a much harder, much better question.

Let's learn

The tool sits inside a company's HR software, right where a manager already goes twice a year to write performance reviews. The manager types a few rough notes about how someone did. The tool pulls in the goals that person set for themselves last cycle. Then it turns both into a paragraph the manager can read, fix, and send.

One note, one goal, one paragraph that outlives the meeting

Before the tool, writing one review by hand took about 35 minutes. A manager with nine direct reports spent just over five hours on that, twice a year, stacked on top of a normal week.

With the tool, the same review takes about 9 minutes. The manager reads the draft, fixes a line, sends it. Twenty-six minutes back, per review, twice a year, for every manager in the company.

Here is the turn. The extra speed is not the problem. The problem is what kind of mistake a faster tool makes, and who ever finds out about it. A company using the tool ran a compliance check on 1,200 drafted reviews and found that 3 in every 100 contained a specific detail, a number, a habit, a missed deadline, that never appeared anywhere in the manager's notes or the employee's goals. The model needed something to say, so it said something plausible instead of nothing.

Ninety-seven drafts out of a hundred were fine. That is a good number on a slide. It is the wrong number to build a decision on, because the three that weren't fine didn't fail loudly. They read like every other line in the review, got signed, and went into a file that follows a person to their next promotion meeting.

The choice I would take back. Early on, the team decided the draft should always come back as a full paragraph, even when the manager's notes were three words long. That felt like the kind choice. Nobody wants a blank box staring back at them. I would take it back. If the notes are thin, the tool should say so, not fill the gap with something that sounds true.

The lesson. We built the tool to save time, and it does. But we measured the time it saved and never measured what kind of mistake it made instead. A faster wrong answer is not progress. It just moves the cost somewhere we weren't looking.

The quarter that changed what Fernhill would ship

You don't need this to answer the question. Read it if you want to feel why the five conditions are specific instead of vague.

Yolanda Pruitt can walk a grocery floor once and tell you which of her nine store managers is about to have a rough quarter, before the numbers say so. She's been a district manager at Halloway Grocers for eleven years, and every March and every September she sits down to write a review for each of them.

Fernhill's drafting tool arrived the spring before last, and for three cycles it was just good. Writing nine reviews used to eat most of a Thursday. Now she typed a few lines about each manager, pulled up their stated goals from the system, and had a draft back in a minute. She read it, tightened a sentence, signed it. Done with all nine before lunch.

She never worried the drafts were wrong in a way that mattered. If a sentence read stiff, she rewrote it. If a line felt generic, she deleted it. Those fixes took her seconds, and she didn't think about them again.

Same tool, two very different mistakes

Then, last September, Fernhill ran a compliance audit across every customer using the tool, comparing 1,200 drafted comments against each manager's own notes, word for word. Yolanda's district turned up in the sample. One review had a line in it: "has missed three inventory deadlines this quarter." Yolanda's notes said nothing about deadlines. She hadn't written it, checked it, or noticed it. It read exactly like every other sentence in the draft, so she signed it in March, and it sat in that store manager's file for six months before anyone looked twice.

He never even knew it was there until the audit surfaced it and HR had to pull the file. He hadn't asked to see his own review since the day he signed off on it, because nobody at Halloway had ever told him he could.

Renata Souza had been the product manager on the drafting feature since before it launched. When the audit results landed on her desk, the header number was fine. Ninety-seven drafts in every hundred matched the notes exactly. Three didn't. She almost let it go at that.

Then she read what the three actually said. Every one was a specific claim, a habit, a number, a missed deadline, that the model had invented to make a short set of notes sound like a full paragraph. None of the invented lines were cruel. That was the part that worried her most. They were small, plausible, forgettable, exactly the kind of sentence nobody double-checks, and exactly the kind that turns up eighteen months later in a promotion committee's file.

She didn't ask engineering to lower the mistake rate. Three in a hundred was already low. She asked a different question: which three in a hundred can we actually afford. A generic sentence in a check-in costs nothing. A made-up one in a review that touches pay or promotion costs a person their record, and they never got a vote on it.

Fixing that one review, once HR found it, took about three weeks and roughly $2,400 in combined HR and legal time to investigate, correct the file, and answer the store manager's questions. On a review that never touches pay, the same kind of fix runs closer to $600, because nobody has to prove anything to anyone; they just cross it out. Multiply the expensive kind by even a small share of drafts across every customer Fernhill has, and the ninety-seven good ones stop being the number that matters.

So the decision Renata made wasn't to shrink the feature. It was to draw a line through it. Full drafting stays on the check-ins and goal recaps a manager reads out loud in the room, where both sides can correct it on the spot. On anything that feeds a pay or promotion decision, the manager writes every specific themselves; the tool only organizes the notes into a structure. Nine reviews a year for someone like Yolanda still take less time than they used to. They just don't take zero of her attention.

The part Renata would take back isn't the launch. It's the early call that the draft should always come back sounding finished. A short, honest "not enough here to draft from" would have cost a manager ten extra seconds. Instead it cost one store manager six months of not knowing what was sitting in his own file.

PICK, one letter at a time

This is a tradeoff question dressed up as a list, so PICK is the tool, not FLIPS. A "how would you measure this" question would reach for LEAD instead.

Try the same five conditions on a claims desk

An auto insurer uses a model to draft the letters an adjuster sends about a claim. Same shape of question: five conditions, different desk, different paper trail.

P. Reject the draft outright the moment it could state fault or a dollar settlement figure the claimant hasn't seen a person confirm. Keep it for routine status updates and document checklists.
I. A wrong line in a routine status update gets caught in one phone call, costs about five minutes. A liability statement that shouldn't have been written gets read into a lawsuit as the company's own words, months later, and by then it's evidence, not a typo.
C. Status-update mistakes are common and cheap, caught constantly, fixed on the spot. Liability misstatements are rare and can't be taken back once they're mailed. Spend the caution where the mistake is permanent.
K. Reject it outright if the draft could read as an admission of fault, if it sets a dollar figure nobody with authority approved, if the claimant would get it before an adjuster reviews every line, or if the model has to guess at a detail the file doesn't contain. Any one of those, and a person writes that letter by hand.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0