Explain the difference between explicit and implicit feedback signals.

Kestrel Loop builds Daily Mix, a playlist every listener gets fresh each morning, tuned from what they've played, skipped, replayed, and saved before. Marisol Cantu has run the recommendations team for four years.

How to answer this, stage by stage

Nobody is grading whether you can define two terms. They're grading whether you'd actually commit to building a system around one of them.

Let's learn

Four years ago, Marisol Cantu watched roughly one in three of Daily Mix's earliest listeners tap a thumbs-up or thumbs-down after almost every song.

Kestrel Loop's Daily Mix is a playlist built fresh each morning for every listener, out of what they've played, skipped, and saved before.

In Daily Mix's first weeks, when only a small group of beta listeners used it, that rating rate held near thirty-four percent. Marisol's team tuned the model against those ratings, and it looked like it was working, because the people who rated were the same people who kept using it.

At its worst: the model spends months confidently serving indie and alternative fans well, since that community rates more than most, while it quietly gets worse for casual pop listeners who never touched the thumbs button at all. Nobody notices until churn creeps up on a group with no complaint anywhere in the data to point to.

What I would leave alone: for a brand-new song with no play history yet, the rare explicit rating still matters a lot, on purpose, since there's no implicit pattern yet for anything else to read.

The lesson: explicit and implicit signals aren't a choice between a good one and a bad one. Implicit is what you build the whole system on, since it's the only one that shows up for everyone. Explicit is what tells you whether you're reading implicit correctly, and that job only works if you keep checking it.

Now here is the same thing as a story

The short version above is what you'd say defending this design to Kestrel Loop's leadership. Read this one for how the gap actually got found.

Marisol Cantu's laptop has had a cracked hinge for two years; she still opens it every night after her kids are asleep to check Daily Mix's numbers before bed.

For most of those four years, the numbers looked fine. The thumbs-rating rate was never huge, but whenever Marisol pulled up the ratio of thumbs-up to thumbs-down, it stayed comfortably positive, and the model kept getting better at pleasing the people who rated.

One kind of signal is honest and rare. The other is constant and needs a translator.

The habit thinned in three beats nobody clocked at the time. First, only the pickiest listeners bothered to rate a song they merely liked. Then, only the sharpest dislikes got a thumbs-down, since a shrug-worthy song wasn't worth reaching for the button. Then, on most mornings, Daily Mix ran its full course for a listener and produced not one single rating, and that stopped being unusual at all.

The trigger was small. A data analyst, reviewing an unrelated dashboard, mentioned to Marisol over coffee, "You know only two percent of sessions ever produce a rating, right? We're tuning a model for millions of people off the opinions of a few thousand who happen to tap a button."

Marisol had heard some version of that line before, in passing, and let it go. This time she pulled the actual numbers, and found the rating rate had drifted from thirty-four percent at launch down to two percent at scale, while implicit signals, skip, replay, save, had sat there with full coverage the entire time, mostly unused for anything but raw ranking.

The step everyone skipped for years is the one in the middle: checking what the skip actually meant.

Run the same drift forward under the old design: the ratings dashboard would have kept looking calm for months, since a small, steady trickle of ratings from the same loyal fans doesn't look like a crisis on a line chart. Nothing about a shrinking sample size sets off an alarm by itself.

Only one signal is both constant and obvious on its own. Everything else needs a second signal to read it right.

With the redesign, Daily Mix's quality read now runs primarily off implicit signals, weighted by session context, and a rolling calibration sample checks those implicit reads against whatever explicit ratings still come in, refreshed every week instead of trusted forever.

Four parts, and the last one, the weekly refresh, is what keeps the whole check from going stale the same way the old rating habit did.

Marisol reran the same three-month window that had gone unnoticed before. This time, the calibration check flagged a drift in how "skip within three seconds" was being read for a newer genre of upbeat, short-intro tracks, catching it in the second week instead of never.

The old design asked a shrinking rating count to keep doing a job it was never built to do at this size. The new one asks the rating for something smaller and more honest: not "is this song good," but "does our reading of a skip still match what a real person meant by it."

I built the thumbs button first because it was the cheapest thing to ship, and for a long while, cheap and correct looked like the same thing. It took a colleague's offhand remark over coffee to see that the button hadn't stopped working. It had just stopped being anyone's.

PICK, in one pageNot a debate over which signal is "better." PICK is what forces you to commit to one and defend the asymmetry.

The recap, one line per letter: position is implicit-first with explicit as a check, impact is the two groups each kind of error hits, cost asymmetry is the invisible misread beating the visible shortage, and kill criteria is the one-in-five disagreement rate that would flip the pick.

And if you want to be sure it really works, try it somewhere elseSame four letters, a veterinary clinic's AI visit summaries instead of a playlist. A different field, the same asymmetry.

Hemlow Veterinary Partners uses an AI tool to draft the after-visit summary a pet owner receives once their appointment ends. Junot Alvear is the product owner for that tool.

Mapped onto PICK: position is the same, build the quality read around implicit signals, specifically how often a vet edits the AI draft before signing it, and how often an owner calls back with a question about something the summary should have already answered, while keeping the rare owner star-rating as a calibration check. Impact: trusting only the star rating tunes the system off the small share of owners who bother to rate at all, usually the ones who had either an unusually great or unusually bad visit. Trusting edits and callbacks alone, with no check, risks reading a vet's stylistic tweak as proof the AI draft was medically wrong, when it might have just been a wording preference. Cost asymmetry: a thin star-rating sample is easy to notice and account for. A misread edit pattern, treating harmless rewording as a real accuracy problem, is invisible and can quietly push the drafting tool in the wrong direction. Kill criteria restates the same one-in-five disagreement rule, checked against a monthly sample of visits with both signals present.

Three of the four paths through this tree produce a signal. Only one of them is explicit, and it's the rarest branch on the page.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "trust implicit signals for coverage, use the rare explicit rating only to calibrate what implicit actually means," and stop.
Cost: there's no budget this quarter for a full calibration pipeline. Say so honestly, and ship a simple weekly spot-check on a small sample first, since a thin calibration still beats none at all.
The model gets better, for real: if skip-prediction accuracy improves, that's still not a reason to drop the explicit check, a better model can still be confidently wrong on the exact case nobody weighed in on explicitly.

Where people run it wrong.
They treat every explicit rating as gospel and every implicit signal as noise, when at scale it's often closer to the reverse.
They read a single implicit action alone as certain, with no session context to check it against.
They build the calibration sample once at launch and never refresh it, so the very thing it's supposed to protect against drifts too.

How to use it live. When someone asks "explicit or implicit," ask yourself which one has enough volume to trust at scale first. That's usually implicit. Then ask what would make you doubt it. That's your calibration sample. Both questions buy a few seconds of thinking time and land on a real, defensible answer.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0