Design the internal triage process for incoming quality feedback.

Caldermill Fabrication machines precision metal parts for industrial equipment makers. LineWatch watches a camera feed on the line and flags anything that looks like a surface defect, a weld crack, a scuff, a dimension that drifted out of spec. Noor Al-Sayed is the plant's only quality engineer, on the floor every day shift.

How to answer this, stage by stage

Nobody is grading how many tiers you can invent. They're grading whether your design actually survives the flag that doesn't fit any tier cleanly.

Let's learn

The clipboard hanging by Caldermill's inspection station has a crack down one corner, from years of being set down too hard between parts.

Before LineWatch, catching a subtle weld crack meant a physical inspection of every tenth part, about twenty minutes per hour of Noor's shift, and still missing the rare ones between samples. LineWatch checks every single part, in real time, and flags anything that looks off in under a second.

Four steps, and the last one is the step nobody designed on purpose.

Now about a hundred and fifty flags land in one shared inbox every night, from four production lines running around the clock. Noor reviews them each morning in whatever order they arrived, which means whichever flag came in last night at 11pm gets seen before one that's been sitting since Tuesday.

The turn: the flags Noor missed weren't the loud, obvious ones. They were the quiet, unfamiliar ones sitting under a pile of routine scuffs and scratches that all looked more urgent simply because they were newer.

At its worst: a genuinely new defect pattern, one LineWatch had never flagged before and wasn't confident about, sits unopened for two weeks under forty routine scuff flags, because nothing separated "unfamiliar and uncertain" from "ordinary."

What I would leave alone: the routine, high-confidence, low-consequence flags, a scuff on a non-critical surface, don't need a person at all most days. A quick same-day batch glance is enough, and building anything heavier for those wastes the redesign's whole point.

The lesson: a triage design isn't measured by how it handles the flags that already look obvious. It's measured by what happens to the one flag that doesn't look urgent yet but is.

Now here is the same thing as a story

The short version above is what you'd pitch to Caldermill's plant manager. Read this one for how the gap actually got found.

Noor Al-Sayed has been Caldermill's only quality engineer for four years, the kind of person who can spot a hairline weld crack from the far end of the floor.

For LineWatch's first year on one line, the mornings were easy. A dozen flags, all reviewed by nine, all closed by lunch.

Four kinds of flag, and only one of them was ever getting lost.

Three more lines came online over eight months. The flag count crept from a dozen a day to over a hundred and fifty a night, and Noor's morning review quietly shrank from "everything" to "whatever's near the top when I open the screen."

The plant manager pulled ten open flags at random for a routine audit and found three sitting untouched for over two weeks, all three of them unfamiliar defect shapes LineWatch had flagged with low confidence, all three buried under newer, ordinary scuffs.

One of these needs speed. The other one needs to be stopped, not sped up.

Caldermill rebuilt the triage around two axes instead of one inbox: confidence, and consequence if the flag turns out to be right. A flag that's both unfamiliar and low-confidence now gets its own queue, held from shipping automatically, routed to engineering the same day it appears.

Four fields on every ticket, decided the instant it's created, not whenever someone gets around to opening it.

Run the same audit forward under the new design: a flag with an unfamiliar shape and low confidence lands in its own held queue the same morning, and engineering sees it before it's ever old enough to look routine.

I merged flagging and triaging into Noor's one job because for a year, on one line, it genuinely was small enough to hold in her head. It took an audit finding three buried flags to see that a design which depends on one person's morning glance isn't a triage process. It's a hope with a job title attached.

SPARK, in one screenNot a feature list. SPARK is what forces a design to survive the day the model is wrong, not just the day it's right.

Six weeks, and the pilot week is where the design actually gets tested before it touches every line.

The recap, one line per letter: situation is one overloaded shared inbox, payoff is trusting a pre-sorted queue instead of checking everything, anchor is the two-axis tier applied the instant a flag is created, risk is the unfamiliar-and-uncertain case getting its own held queue, and keep out is root-cause tagging waiting for a later version.

And if you want to be sure it really works, try it somewhere elseSame five letters, a permits office instead of a factory floor. Nothing else about the two jobs is alike.

Fenwick Creek's municipal permits office uses an AI tool that flags incomplete or risky building-permit applications for a human reviewer before approval. Renata Vasquez has processed permits there for six years.

Mapped onto SPARK: situation is the same shape, every flagged application landing in one shared review queue in arrival order. Payoff is reviewers trusting that queue already separated the routine missing-signature cases from the ones that actually need a real look. Anchor: tier every flagged application by how confident the tool is and how serious the consequence of a wrong approval would be, same two axes, a different building entirely. Risk: an application involving a genuinely new zoning situation the tool has never seen, flagged with low confidence, gets its own held tier instead of vanishing under routine paperwork flags. Keep out: don't try to auto-approve the routine tier yet, day one only sorts, it doesn't decide.

Three real ideas, and none of them belong in the version that ships first.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "tier by confidence and consequence the instant a flag lands, and give the unfamiliar-uncertain case its own held queue," and stop.
Cost: building automatic routing takes an extra sprint the team doesn't have yet. Say so, and ship the tiers as a manual dropdown first, since a slower tiered process still beats a fast untiered one.
The model gets better, for real: if LineWatch's accuracy improves overall, the tiers still matter, because a rarer miss is an even easier one to mistake for routine and lose in the queue.

Where people run it wrong.
They build one shared queue and call reviewing it in order "triage."
They let low confidence quietly mean low priority, without ever deciding that on purpose.
They try to build root-cause diagnosis before the basic tiering even ships.

How to use it live. When someone asks you to design a triage process, ask yourself first: what happens to the flag the model is least sure about. If your design doesn't have a clear answer for that one, it isn't done yet.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0