How do you specify evals for safety and policy compliance separately from quality?

How to answer this, stage by stage

Seven moves. The middle three, naming who each eval serves, what a good score would let slide, and who can't push back, are where the real answer lives.

Let's learn

Here's what happens when an answer sounds right and breaks a rule at the same time, and only one of those two things shows up on the scoreboard.

Say a company builds HireLens. It records a job interview, writes it out word for word, then drafts a short analysis for the recruiter: what the candidate did well, what's worth a second look, a recommendation.

Before the tool, a recruiter wrote those notes by hand. Twenty minutes after every interview, five interviews a day, that's about an hour and forty minutes just writing up what already happened.

Now HireLens drafts the same notes in under a minute. The recruiter reads it, edits a line or two, and moves on. The team built a hundred-point rubric to grade every draft: is it accurate, is it clear, is it useful, and one line, worth ten of the hundred points, checking that it never names a protected characteristic.

For over a year the average score sat at ninety-two out of a hundred, so the rubric looked like it was working. Here's the turn. Those ten points were never the real problem. The real problem is that ten points can get outvoted. A draft that infers a candidate's caregiving status can still write that inference in clean, confident, well-formatted prose, and clean, confident, well-formatted prose is exactly what the other ninety points reward.

At its worst, this lands on a candidate who never gets a callback, filtered out by an inference nobody meant to make and nobody checked, while the dashboard upstairs still reads ninety-two out of a hundred and calls the quarter clean.

What I would leave alone. Tone and formatting checks: does the summary read warm enough, is it the right length, does it follow the house style. None of that touches a protected characteristic. Folding those into the same forgiving quality score is fine, they were never the problem.

The lesson. A rule that shares a scoreboard with something else can be outvoted by it. If you want a rule that never loses, don't let it compete.

Now here is the same thing as a story

Read the short version above if you want it fast. Read this one when you want to feel why the fix matters, not just know what it is.

Miriam Solano can read a hundred-point rubric and tell you inside a minute which line is doing real work and which one is decoration. She's the eval lead for HireLens, Solvane's interview-analysis product, and she wrote the original rubric herself during the pilot, eighteen months ago, with three early customers watching every score.

For the first several months, that rubric was the whole job. Miriam read every flagged "concern" draft, line by line, every single week, checking that nothing had drifted. The average score climbed to the low nineties and held there. She kept reading anyway, out of habit more than anything.

Then she started skimming instead. The score held. Weekly checks became monthly. By the time the average had sat at ninety-two for a year, she was glancing at the dashboard number and moving on to the next fire.

Nobody told her this was risky. It looked, every single week, exactly like it was working.

Then a new recruiter, three weeks into the job, asked her a plain question in the hallway. "Why did HireLens flag this candidate's availability as a 'moderate concern' when he never once said anything about his schedule?" Miriam pulled the transcript. Near the top of the call, making small talk, the candidate had mentioned he had two young kids. HireLens's analysis, in the scheduling section, had written: "Candidate indicated potential caregiving responsibilities; may affect flexibility for early or on-call shifts. Moderate concern." Ninety-six out of a hundred on the quality rubric. Clear. Well-formatted. Relevant to the hiring call, on paper.

Same report, same outcome. Only one of them can change the setting.

Miriam pulled the last quarter's flagged drafts that weekend, fifty of them, out of four hundred interviews HireLens had processed. Six of the fifty contained something like it: a caregiving inference, an age assumption dressed up as "recent graduate, may need more ramp time," a comment on "communication clarity" that tracked suspiciously well with non-native accents. All six had scored above ninety on the quality rubric. Their average was ninety-three, a full point above the rubric's own yearlong average. The violations weren't dragging the score down. They were hiding inside it.

I would take back the decision to put the safety line inside that same hundred-point score at all. Not the rule itself, the rule was right. The scoreboard it lived on was wrong. Ten points is a discount a good sentence can always afford to pay.

Run the quarter again with a pass or fail gate sitting in front of the quality score instead of inside it. All six of the backlog's violating drafts get caught before they'd have reached a recruiter's inbox. Going forward, the gate catches about one draft in forty-five before release, each one rewritten before anyone sees it, and the flag-rate gap between candidates who mentioned family or health topics and everyone else starts closing toward the baseline instead of sitting there unmeasured.

The thing I'd tell myself, eighteen months back: a score that averages two questions into one number can only ever answer the easier one.

GUARD, aimed at one merged score

This is a risk question, so the framework is GUARD. "How do you split two evals" reads like a spec-writing question, which is exactly why nobody flagged the merged rubric as the actual gap until a new hire's plain question found it.

The box that should sit here, and doesn't.

And if you want to be sure it really works, try it somewhere else

A city permits office uses an AI tool, PermitScope, to review building-permit applications and draft a risk note for the inspector. Different building entirely, same five letters, same trap.

G, groups. Inspectors, and whoever owns PermitScope's rubric, enforce the rule that a risk note can never cite an applicant's neighborhood, income level, or the look of their existing structure as a reason for extra scrutiny. Applicants, especially first-time homeowners and small contractors, are who it protects.
U, unequal. A note reading "modest existing structure suggests owner-builder work, recommend closer review" doesn't land evenly. It lands hardest on lower-income applicants whose homes already look more improvised, and it slows their permit exactly where it should move fastest.
A, ability to contest. An applicant reading a "needs review" notice has no way to see the risk note behind it, let alone the line that triggered it. By the time she could ask, the permit has already lost a filing season.
R, reduce. Fifty adversarial applications, a modest home, an unusual-sounding name, a low-income zip code, and the model has to write its risk note using only the actual code violation, never the property's appearance or the applicant's background. Zero tolerance, same shape as the hiring gate.
D, detect. Every risk note gets checked against a by-neighborhood flag-rate dashboard, not just the office's average review time. A gap opening between neighborhoods with similar violation rates blocks the rubric until it's fixed.

Flashcards (click a card to flip it)

Check yourself Score: 0 / 0