You're Meta's PM for AI chat. Define success and goals

[INTERVIEWER] You're Meta's PM for AI chat. Define success and goals. You're Meta's PM for AI chat. Define success and goals. And here is the trap sitting right in front of you: defining success as message count. Because message count rewards a chatty bot that wastes people's time, and worse, it goes up precisely when the assistant is bad and the user has to keep asking again.

The strong answer defines success by user value first, useful replies and retention, then connects it to the actual business, and builds the funnel from awareness all the way to habit. Value first, business second, and never message count. The interviewer is testing whether you can pick a goal that stays honest even within the intense focus on engagement at Meta.

It is easy to reach for a big engagement number at Meta scale. The skill is choosing a metric that only moves when the user is genuinely better off, and then knowing which stage of the funnel is actually leaking so you know where to push. Get this right and you show you can set direction for a product, not just track it.

Let us build it out. First, clarify the scope, because Meta AI lives in several places: WhatsApp, Messenger, Instagram, and as a standalone app. Is the goal engagement inside those apps, a new standalone assistant, or both? And what is the business rationale: retention of the family of apps, advertising adjacent value, or a strategic bet on owning the assistant?

I would define success for Meta AI as an assistant embedded across the family of apps, leading with user value before tying it to the business. A valueless engagement metric is exactly how you build something people use twice and then abandon. State your read, then build on it. For the North Star, I would choose weekly active users who get a useful answer, where useful is instrumented rather than assumed.

I define useful as a query ending in a positive signal. The user acts on the reply, does not immediately rephrase or abandon, gives a thumbs up, or continues the task. Count it as useful query weekly actives, plus the rate of useful queries per user. Here is why this beats raw messages, and I would say it out loud.

Message count goes up when the assistant is bad, because a confused user keeps asking the same thing again in five different ways. So a bot that pads its message count with filler looks like a winner on messages and a loser on usefulness. My North Star rewards the assistant actually helping, and it refuses to reward the assistant wasting someone's afternoon.

That distinction is the core of the answer. Now, the goal for Meta is not chat for its own sake. It is retention and monetisation of the family of apps. So I add secondary metrics, kept explicitly secondary. Does Meta AI usage lift overall app retention and time, does the assistant make WhatsApp and Instagram stickier? And does it open new value, better content discovery, and eventually commerce or advertising adjacent help?

I keep these below the user value North Star on purpose, so I am never caught optimising engagement against the interest of the user. The order matters: user value is the master metric, business value is what it should produce downstream. Now map the journey and put a metric on each stage. The North Star moves through the funnel, so you do not push it directly.

Awareness, or entry: users who see the Meta AI entry point, the search bar, or the Meta AI mention tag. Metric: entry point impression and tap rate. Activation: users who send a first real query and get a useful answer back. Metric: first query success rate. This is the critical step, because a bad first answer kills everything downstream. Nobody comes back after the assistant fumbles its one shot.

Engagement: useful queries per active user per week, which is the North Star rate. Retention: week one and week four return rate of users who activated, the real proof of value. And referral, or spread: in a social context, sharing an AI reply into a chat, which is the specific distribution advantage for Meta. Then I name which stage I would attack first, and here is the specific read for Meta.

Awareness is nearly free. The entry point is bolted into apps with a billion users, so the leak is almost never reach. It is first query success and retention. That is where I would push. Because this is a social AI product for all ages, I add guardrails. Safety and harmful content rate. Hallucination rate, because people will absolutely ask it factual questions and trust the answer.

A time well spent signal, so we are not quietly building an addictive time sink. And latency. Plus one more that is easy to miss, a cannibalisation check. Meta AI should add to app value, not just shuffle engagement around the family of apps. So I watch overall app retention as a guardrail against a vanity metric inside the assistant that is actually stealing from Instagram to feed the chatbot.

Let me put numbers on it. North Star: useful query weekly actives, useful meaning the reply gets acted on or receives a thumbs up with no immediate rephrase. Say Meta AI reaches 500 million weekly actives. Distribution is easy at Meta scale, but only 45 percent hit a useful query, so 225 million useful query weekly actives. Now, because awareness is basically solved by embedding into WhatsApp and Instagram, the growth lever is activation and retention, not reach.

So I attack first query success rate, currently the biggest leak, by improving answer quality on the top query intents. These are the handful of things people actually ask most. Guardrails: overall WhatsApp retention must not drop, no cannibalisation, harmful content rate stays under threshold, and the helpfulness survey stays positive. Target for the quarter: lift the useful query rate from 45 to 55 percent and hold week four retention flat or rising, all without touching message count, which we explicitly do not optimise.

And that last clause, which we explicitly do not optimise, is the sentence that tells the interviewer you understood the trap and stepped around it on purpose. Expect the interviewer to squeeze on the business tension. They will ask why you keep saying user value over message count when Meta makes money from engagement, and whether you are fighting the company model.

And you need a clean answer, not a flinch. The answer is that useful engagement and profitable engagement are the same thing over any horizon longer than a quarter. A user who gets genuinely helpful answers stays, opens the app more, and stays reachable for the ads business. A user farmed for hollow message count churns, and a churned user monetises at zero.

So user value is not in tension with the business model. It is the durable foundation of it. I would frame the North Star as protecting long term engagement against a short term vanity metric that ultimately degrades the core product. The second push you should expect is how you would even measure useful, since that sounds subjective. So I would get concrete about instrumentation.

Useful is a composite of behavioural signals you can log. No immediate rephrase within the session, a copy or a share or a task continuation, dwell time that suggests the answer got read rather than dismissed, and a thumbs up when offered. Then I would validate that composite against a periodic human rated sample. A few thousand conversations scored by raters for actual helpfulness, to confirm the behavioural proxy tracks real usefulness and recalibrate when it drifts.

Being able to turn a soft word like useful into a logged, human validated metric is exactly the instrumentation credibility this round is grading. Here is what makes them lean in. First, you refused message count as the North Star and explained why it is perverse, specifically that it rises when the bot is bad. Naming the mechanism beats just naming a better metric.

Second, you sequenced user value first and then the Meta business layer, app retention and distribution through sharing, rather than picking one and ignoring the other. And third, you read the specific advantage for Meta, free awareness at a billion user scale, and correctly attacked activation and retention instead of wasting effort on reach that is already solved. Reading the specific company situation is what makes it a strong answer and not a generic one.

The traps. One, picking messages per user or DAU, which rewards a chatty, time wasting assistant and is the exact trap the question sets. Two, listing metrics with no funnel and no statement of which stage is the real leak, so you sound like you are reciting rather than diagnosing. And three, ignoring cannibalisation, so Meta AI wins by moving engagement around the family of apps without adding a single unit of new value.

At Meta, that is a very easy and very hollow way to look successful. So the shape is clear. Clarify the product and its business context, pick a North Star built on useful replies and instrument the definition of useful concretely, keep business metrics secondary, map the funnel from awareness to activation to engagement to retention and name the real leak, and guard against cannibalisation.

Carry this into the room. Define success by useful replies and retention, never message count. Then map the funnel from awareness to activation to engagement to retention, and attack the real leak. At Meta scale that is activation, not reach.