Critique a product decision that treats compliance as a launch-day task.

Here is what happens the first time a checklist item turns out to be decoration. Thistle Home matches customers with contractors, plumbers, electricians, HVAC technicians, using a model that also checks whether a contractor's license covers the job. Renata Alcaraz is the product manager who pulled the team's launch checklist apart after a competitor marketplace got fined over a bad match.

How to answer this, stage by stage

Six stages. A critique question wants you to name the exact gap, not just say "this is bad."

Let's learn

Say a home-services marketplace builds a model that matches customers with contractors, and also checks whether the contractor's license actually covers the job being booked.

For a long stretch, the team's nine-item launch checklist looked fine on paper. Every AI feature shipped with all nine boxes checked, including "Compliance review." Nobody in leadership had reason to look closer, because the box was always checked.

What actually happened, quietly, over about a dozen launches: engineers who used to flag last-minute regulatory concerns in the team channel learned that doing so only got them blamed for holding up a date already promised to the board. So they stopped raising them early, and concerns started surfacing only after a customer complained.

At its worst: a contractor gets matched to an electrical job through a state where their license had quietly lapsed two months earlier. Nobody inside the company catches it. The customer does, files a complaint, and Thistle finds out the same way everyone else eventually will, from the outside.

What I would leave alone: the other eight items on the checklist, design review, load testing, rollback plans, are genuinely doing their job. The fix here is narrow, one item, not a rebuild of the whole process.

The lesson: a checklist item with no owner and no criteria isn't a safeguard sitting dormant until you need it. It's already failing, silently, every single time it gets checked.

Now here is the same thing as a story

The short version above is what you'd say defending this critique out loud. Read this one for how Marisol actually found the gap.

Renata Alcaraz had run product at Thistle Home for three years by the time a competitor marketplace, in a different city, got fined after matching a customer with an unlicensed electrician through its own AI system. It hadn't happened to Thistle. It happened to a company just like it, and that was enough to make leadership ask Marisol to check their own process.

Five steps, and the third one, where a real gate should have sat, was never actually built.

She pulled the last dozen launch checklists and found the same pattern on every single one: item 9, compliance review, checked, dated the same day as ship, no name next to it, no written criteria anywhere in the ticket.

She talked to the engineers who'd flagged concerns in the past. More than one said some version of the same thing: raising something late got you blamed for the delay, so eventually you just stopped raising it early and let it surface however it surfaced.

One of these two people has a lever. It just happens to be the one furthest from where the actual harm lands.

She sorted the people exposed by the gap onto one page: who has the power to delay a launch, and who actually carries the risk if the match is wrong.

The people with the least power to slow anything down were exactly the ones carrying the most risk.

She rebuilt the one broken item, not the whole checklist, around what an actual review requires.

Four parts. The old checklist item had reliably had exactly none of them.

Same feature, same team. The only thing that moved was when someone with real power actually looked.

With the fix in place, a feature can't enter the freeze branch without a signed, criteria-based review attached to its ticket, six days before ship on average now instead of zero.

Four numbers, and the first two are the pair that would have caught the old pattern within a single quarter.

The old checklist asked whether nine boxes were checked. The new one asks whether the ninth box ever had the power to say no.

We put compliance last on the list because, for a long while, the features shipping genuinely were low-risk, and last felt harmless. It took a competitor's fine, not our own incident, to see that the item had quietly become a rubber stamp long before the risk caught up to it.

GUARD, applied to a checklist instead of a modelNot a policy lecture. GUARD is what forces you to name who's actually exposed by a process gap, not just that the gap exists.

The recap, one line per letter: groups is the contractor and the customer exposed by the gap, unequal is that solo contractors and licensed-trade customers carry the risk with the least power to delay anything, ability to contest is the missing pre-launch lever, reduce is a named, criteria-based, freeze-blocking review, and detect is tracking sign-off timing every sprint, not just after an incident.

And if you want to be sure it really works, try it somewhere elseSame five letters, a code-review SaaS instead of a home-services marketplace. This time the gap sits in a different kind of checklist entirely.

Driftwood CI runs an AI reviewer that flags security and licensing issues in customers' pull requests before merge. Jaan Kask manages that reviewer feature.

Mapped onto GUARD: groups is the customer whose code ships with a flagged issue nobody actually looked at, and the Driftwood engineer who owns the reviewer's rule set. Unequal is that customers on the free tier, who can't pay for a human escalation path, absorb the risk of an unreviewed flag more than paying customers with a dedicated account manager. Ability to contest is that free-tier customers have no way to ask a person to double-check a flag the AI reviewer marked "low confidence" and silently downgraded. Reduce is requiring a human glance at any flag below a stated confidence floor before it's downgraded, regardless of tier. Detect is tracking what percentage of low-confidence flags get a human look before being auto-dismissed, watched weekly instead of discovered in a support escalation.

A different product, the same four missing parts, this time behind a pull request instead of a contractor match.

Swap the trigger and it still runs.
Speed: an interviewer caps you at a minute. Say "the checklist item has no owner, no criteria, and happens after it could still stop anything, fix all three," and stop.
Cost: leadership says a full pre-freeze gate is too slow for this quarter's launch cadence. Start with the one riskiest feature category, licensed-trade matches, and expand from there.
The model gets better, for real: if the matching model's overall accuracy improves, that doesn't fix a review process with no owner and no criteria. A more accurate model with an unowned rubber stamp behind it still has an unowned rubber stamp behind it.

Where people run it wrong.
They rebuild the whole checklist when only one item was ever broken.
They add more checklist items instead of giving the existing one an owner and real power.
They treat "we do a compliance review" as true because a box gets checked, without ever asking when, by whom, or against what standard.

How to use it live. When you're handed a document like this, look for exactly three things: a named owner, written criteria, and the power to actually block something. Say out loud which of the three is missing before you say anything else.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0