Critique an internal tool built with no user research because the users sit nearby.

Interviewer's question: "Critique an internal tool built with no user research because the users sit nearby." Renata Szabo built a structural-detail-checking AI plugin for Loom & Lattice Architecture's CAD software, testing it only with the senior architects who work down the hall from her at the main studio.

How to answer this, stage by stage

Nobody is grading whether you can say "do more user research." They're grading whether you can name exactly whose workflow got left out, and how you'd know before it costs you.

Let's learn

Picture a tool that passed every test its builder ever ran on it, and still failed the first week it left the building.

Loom & Lattice Architecture's structural-detail-checking plugin sits inside their CAD software and flags likely errors in a drafted detail, a missing dimension, an unsupported span, before it goes to review. Renata Szabo built it and tested it entirely with the senior architects who work down the hall from her at the main studio.

By every measure Renata had, the tool was a success. Main-studio adoption was strong, feedback was glowing, and nobody down the hall had a single complaint.

Same tool, same company. Only one of these two people ever got a say in how it works.

The turn: "no complaints" was never proof the tool worked everywhere. It was proof the tool worked for the one group who'd been consulted, and everyone else had simply never been given a way to say otherwise.

At its worst: the satellite office quietly stops using the plugin altogether, reverts to the old manual detail-check process, and nobody at headquarters notices for months, because a tool nobody uses generates no complaints either.

What I would leave alone: the plugin's core detail-checking logic for standard, non-preservation work doesn't need to change at all. The gap isn't in what the tool does well, it's in who it was ever tested against.

The lesson: "the users sit nearby" describes a sample of convenience, not a research pass, and the two look identical right up until someone two floors away actually tries to use the thing.

Now here is the same thing as a story

The short version above is what you'd say in a design review. Read this one for how the gap actually surfaced.

For three months, Renata Szabo's plugin was the best-reviewed internal tool the firm had ever shipped. Then someone two floors away, in a different city, actually tried to use it.

Three quiet months of glowing reviews, and it took one new hire's honest question to end them.

A newly hired architect at the satellite office, still new enough to ask questions nobody else thought to ask anymore, raised her hand in a training session: "Why doesn't it recognize any of our preservation annotations? It just flags all of them as errors." Renata didn't have an answer, because she'd never once watched anyone at that office actually draft a detail.

Renata pulled the adoption numbers by office that same afternoon. Main-studio usage sat at 91 percent, exactly as glowing as every internal report had said. Satellite-office usage sat at 12 percent, a number nobody had been tracking separately, buried inside a company-wide average that looked perfectly fine.

Renata's fix wasn't a bigger model or a longer feature list. She sat with two drafters at the satellite office for fifteen real minutes each, recorded, watching them work through an actual preservation detail from start to finish, and added a second annotation profile the plugin could recognize. Satellite usage climbed back above 70 percent within a month.

GUARD, in one screenNot a diversity checklist. GUARD is what makes an invisible research gap show up before production does.

Four requirements, and none of them are a hallway conversation with whoever's around.

The recap, one line per letter: groups is the main studio against the satellite office and junior drafters, unequal is the preservation-annotation workflow the harm actually lands on, ability to contest is a satellite architect with no lever at all, reduce is a real fifteen-minute shadowing session per office and band, and detect is adoption tracked separately by office.

And if you want to be sure it really works, try it somewhere elseSame five letters, a regional grocery chain instead of an architecture firm. Nothing else about the two jobs is alike.

Pinemont Grocers built an internal AI tool for shelf-stocking schedules, researched only with staff at the flagship downtown store, never with staff at the chain's rural stores.

The two closest to the builder got asked. The two farthest away never did, regardless of seniority.

Mapped onto GUARD: groups is flagship downtown staff, tested and consulted, against rural-store staff, never asked at all. Unequal is that rural stores restock on a completely different weekly delivery schedule the flagship store never deals with. Ability to contest is a rural stocker with no lever, since the tool shipped "done" the moment downtown staff approved it. Reduce is a real recorded shadowing session at one rural store before calling research finished. Detect is tracking schedule-override rate by store type, since rural stores overriding the tool constantly is the exact tell nobody was watching for.

Four inputs, and every single one came from the same hallway.

Swap the trigger and it still runs.
Speed: an interviewer caps you at thirty seconds. Say "the users down the hall were a sample of convenience, not a research pass, and the real gap shows up as adoption cratering in the group nobody asked," and stop.
Cost: if a full research pass genuinely isn't affordable before launch, at minimum ship with adoption tracked separately by group from day one, so the gap surfaces in weeks instead of months.
The model gets better, for real: if the plugin's overall accuracy improves, the gap doesn't close on its own, since a better model trained on the same narrow group just gets more confidently wrong about the workflow it never saw.

Four steps, and the third one is the empty space where a real appeal should have lived.

Where people run it wrong.
They treat "nobody down the hall complained" as proof the tool works everywhere, instead of proof it works for the one group who got a say.
They watch one company-wide adoption number instead of breaking it out by the exact groups who might differ.
They build a feedback box instead of a real lever, so the group with the different workflow can only vote by quietly disappearing.

How to use it live. When someone tells you research wasn't needed because "the users sit nearby," ask yourself first: nearby to whom, and who does that leave out. Name that group specifically, and the critique writes itself.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0