How would you structure the AI PM function at a 200-person company launching its first AI features?

Quillfen Benefits builds the software other companies use to run open enrollment for their own employees. This fall it is shipping its first two AI features: Hearthway, a chatbot that answers plan questions inside the enrollment portal, and Coverwise, a tool that ranks an employer's health plans by which one will actually cost an employee the least over the year. Halfrida Longacre, Quillfen's VP of Product, has to tell the CEO how many AI product managers to hire, and how to organize them, before open enrollment opens.

How to answer this, stage by stage

Nobody is grading whether you can say "AI PMs should sit close to engineering" and sound thoughtful. They're grading whether you can turn "how many do we need" into a real equation, a sourced number, and a range you'd defend when someone pushes back on it.

Let's learn

Five steps. The fourth one, checking the estimate against a threshold before it ships, is the step this whole question turns on.

Quillfen's platform runs open enrollment for other companies' employees: pick a plan, sign up dependents, done. Hearthway is a chatbot inside that portal that answers plan questions in plain language. Coverwise goes further. Tell it your dependents and how often you expect to see a doctor, and it ranks the employer's plans by which one will actually cost you the least over the year.

Before Coverwise, picking a health plan meant comparing four or five carrier PDFs by hand, each one thirty or so pages of deductible tables and copay tiers. Most people picked the plan a friend recommended, or whatever they had last year, because reading four PDFs during open enrollment, on top of a full time job, is genuinely miserable. With Coverwise, an employee answers six questions and gets a ranked list in about ten seconds, with an estimated yearly cost next to each plan.

Here's the turn. The real risk with Coverwise was never a single wrong estimate. Employees make this decision once a year and mostly don't check it against anything. The real risk is a whole plan year of employees trusting a number that stopped being true the moment their carrier changed the plan's rules, with nothing built to catch it.

What it costs at its worst: a family enrolls in a high deductible plan because Coverwise says it will cost them $640 for the year. The real number, under the plan's new rules, is $2,150. They find out in March, when the bills start arriving, not in November when they could have picked differently.

What I would leave alone: the sales team's small AI tool that drafts outbound emails. It shares no eval surface with anything else, touches no personal health data, and one person already owns it as a small part of their role. There's no shared ownership problem there, because there's nothing shared.

The lesson: a feature whose model only touches a chat window can get away with loose ownership of its own quality checks. A feature whose model tells someone how to spend real money for a year needs a named owner for that check, the day a second feature starts sharing its infrastructure, not after.

Now here is the same thing as a story

The short version above is what you actually say in the room. Read this one when you want to feel exactly what one skipped ownership decision nearly cost.

Halfrida Longacre can tell you, without opening a spreadsheet, which of Quillfen's customers are about to have a rough open enrollment. Nine years running product for benefits software will do that. She built Quillfen's product org from four people to twelve, and every one of those twelve knows her rule: nothing ships without someone's name next to it.

Hearthway shipped first, back when it was Quillfen's only AI feature. A chatbot, tucked into the enrollment portal, answering things like "does my plan cover physical therapy" in a sentence instead of a PDF. Zosime Cinderbourne owned it, and for its first two quarters, the shared work behind it, keeping its eval set current, watching for drift, was light. Maybe five hours a week. Halfrida decided, reasonably, not to name anyone as its formal owner. One feature, five hours, didn't seem worth carving out of anybody's role.

The workload doubled. The ownership stayed exactly where it started: nowhere in particular.

Then Coverwise shipped. Oriana Quintero owned it, ranking plans by real yearly cost instead of just answering questions about them. Both features leaned on the same eval harness, the same golden set of test cases, the same drift monitoring dashboard. The shared work didn't stay at five hours. It doubled to 16. And ownership stayed exactly where it had been: nowhere in particular. Zosime assumed the platform squad had eyes on it. Oriana assumed Zosime did, since Hearthway had been there first. The platform squad assumed a PM owned the business call of when an eval set needed refreshing, since that wasn't an engineering decision.

For most of a year, that gap didn't matter. Then two of Quillfen's largest carrier partners did what carriers do most years: they restructured their deductible and copay tiers for the new plan year. About 30 percent of the plans in Quillfen's system changed a tier that cycle, an entirely ordinary amount of carrier churn. Coverwise's cost estimator had been built and checked against the old plan year's numbers. Refreshing that check before the new enrollment window opened was nobody's named job, so nobody did it.

Nobody decided, on any single day, to skip the refresh. It just never became anyone's explicit job.

A week before Coverwise was due to go live for Quillfen's biggest customer, a manufacturing company with 4,200 employees, Oriana ran a routine spot check. Out of habit, not worry. She picked a new high deductible plan at random and ran a test family through it. Coverwise said $640 for the year. Oriana pulled the carrier's actual new plan documents and did the math by hand. $2,150.

The decision Halfrida would take back sits in a meeting eight months earlier, the week Hearthway shipped alone. Someone asked whether the shared eval and drift work needed a formal owner. The room said no, reasonably: one feature, five hours a week, and naming an owner for something that small felt like process for its own sake. Nobody wrote down that the answer would change the day a second feature started sharing the same infrastructure.

Run the eight months again, with a named owner for that shared work from the day Coverwise entered the roadmap, even at a quarter of someone's time. That person's job includes one specific line: refresh the golden set against each carrier's new plan year before the next enrollment window opens. The carrier restructuring still happens. The same 30 percent of plans still change tiers. But the golden set gets refreshed in October, three weeks before rollout, and the $2,150 shows up on a screen in a test environment instead of in a family's mailbox in March.

One design leaves the truth of a plan's cost drifting for months, checked only if someone happens to look. The other checks it on a schedule, out loud, every plan year, whether anyone remembers to ask or not.

What I'd tell myself, sitting in that meeting: five hours a week is a real number, but it isn't the number that matters. The number that matters is how many people would trust the output if that work stopped happening. For a chatbot answering questions, that's low stakes. For a tool telling someone what a year of health care will cost them, it's the whole point of the product.

BOUND, or turning a headcount guess into a number you can defend

Not a way to sound thoughtful about org design. BOUND is what forces "how many AI PMs do we need" into an equation with sourced numbers, instead of a headcount request nobody can trace back to anything.

The five moves, in order. The fourth one, the check against the rest of the company, is the step a rushed answer skips.

Four things both features lean on. None of them belonged to either applied PM alone, which is exactly why they went unowned.

Three things worth naming directly, since this is where the real judgment sits. The AI-specific failure mode here is silent distribution shift: the cost model doesn't break or throw an error, it just keeps confidently answering a question whose real answer changed underneath it, the moment a carrier restructures a plan's deductible and copay tiers. The guardrail is a named owner and a threshold, not a vibe: Coverwise only ships a plan year refresh once its estimates clear the 95 percent, within 10 percent bar on a golden set, checked again every plan year. That's a real trade-off, accepted on purpose: keeping the platform role part time instead of hiring it dedicated on day one saves real headcount budget, and it accepts a small amount of risk that the shared work grows faster than expected before anyone notices the hours climbing. The alternative Halfrida's team considered and turned down was one shared "AI product lead" across both features instead of two applied PMs. It lost because Hearthway and Coverwise don't share a stakeholder, an engineering squad, or a launch date, and one person switching between actuarial cost modeling and chatbot content would have been slower at both than two people who could each go deep on one.

And if you want to be sure it really works, try it somewhere else

Same five letters, a fishing fleet instead of a benefits portal, and this time the shared work that goes unowned isn't a golden set, it's a camera feed.

Different building, same shape of decision. A golden set of plan and member combinations becomes a golden set of labeled catch photos.

Fathomwell Fisheries runs a fleet of nine trawlers and is shipping its first two AI features this season: a catch yield predictor that tells a captain where fish are likely to be, and a bycatch tool that flags a protected species in the net from onboard camera footage before the haul finishes. Isbeau Fairleigh, VP of Product there, is asking the same headcount question Halfrida asked, on a much smaller boat.

Run BOUND on it. Break it down: one applied AI PM per feature, plus a shared role for what both features lean on, the camera and sonar pipeline, the species eval set, the offline first sync layer boats need since they lose signal at sea. Own the numbers: two applied PMs, and a time audit puts the shared work at 14 hours a week today, under the 20 hour threshold, so it folds into a third PM's role at half time. Use a range: three AI PMs now. A third funded feature, predicting engine failures from sensor data, is already on next season's roadmap, and it would add about 9 more hours a week of shared work, crossing the threshold and turning the platform role dedicated. Four AI PMs, if that ships.

Same two checks, three different companies. Fathomwell's dot sits in a spot that would look alarming at Quillfen, and is the honest answer anyway.

Direction, the same swing assumption as Quillfen, just smaller: how many features are genuinely funded, two or three, is still the one number that moves the answer, this time by one person instead of two.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: one applied AI PM per feature that's actually shipping, plus a shared role that only becomes dedicated once its hours cross a real threshold, not a guess.
Cost: no budget for a third hire this quarter. Fold the shared work into one applied PM's role at a stated 50 percent, name it in writing, and revisit the split the moment a second feature enters build.
The model got better, for real: say a cheaper, more accurate model cuts Coverwise's build cost in half. The headcount math barely moves, since the shared eval and drift work doesn't shrink just because the model got better. Someone still has to keep checking whether it's right for this year's plans.

Where people run it wrong.
They hire one AI PM per idea on the roadmap instead of per feature that's actually funded and staffed.
They let "everyone owns it" stand in for a real owner on shared model quality work, which in practice means no one does.
They size headcount off total reach or ambition instead of checking it against the rest of the company's PM and engineering numbers.

How to use it live. Before naming a number, ask yourself one thing: "how many of these AI features have their own engineers already writing code, right now, not just a slide in a roadmap deck?" That question alone turns a wish list into a real count.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0