How do you avoid the trap of building an AI feature that demos well and gets no repeat usage?

Larderwell is a grocery delivery and personal-shopping app. Its AI plans a week of meals and fills your basket for you. Sixten Groenewald is the product manager who owns that AI shopper. Anzhelika Farkas, Larderwell's VP of Growth, championed the app's newest feature, Surprise Basket, off the strength of one incredible demo. Dulcie Holmqvist, a nurse who has planned her family's meals by hand for six years, tried it once.

How to answer this, stage by stage

Nobody is grading whether you'd feel bad for Dulcie. They're grading whether you can turn "avoid the trap" into a specific bar you'd actually enforce.

Let's learn

Larderwell is an app that plans your week of groceries and fills the basket for you, so you're not building the list from scratch every Sunday.

Four things Dulcie checks every single week, without fail, because missing even one of them isn't a small mistake for her family.

For six years, Dulcie Holmqvist did that planning herself. She's a nurse, she works rotating shifts, and every Sunday she'd spend about ten minutes with her own list: nothing with peanut in it, nothing that pushes her past a hundred and forty dollars, and mostly the same dozen meals on rotation, because rotation is what actually gets cooked on a Tuesday night after a twelve-hour shift.

For years, Larderwell judged a new feature by one number: how many shoppers try it in week one. That worked fine, because most of what they shipped was small and useful, things like The Regulars, a feature that quietly reorders whatever a household actually buys most weeks. Nineteen percent of shoppers tried The Regulars in its first week. Of those, sixty-eight percent were still using it two weeks later.

Then Larderwell shipped something genuinely new. Surprise Basket is an AI feature that dreams up an unfamiliar dinner every week and fills your basket with what it needs, ingredients you'd never have bought yourself. Sixty-one percent of shoppers tried it in its first week, more than three times The Regulars' number. Anzhelika Farkas put it up on the all-hands screen and called it the best first week any feature had ever had at Larderwell.

Here's the turn. A feature a lot of people try once is not the same thing as a feature people need. Of the shoppers who tried Surprise Basket in week one, only six percent ever opened it again inside two weeks.

The Regulars checks all four of these, every single week. Surprise Basket's model was never told to.

Here's why. Surprise Basket's model was scored on one thing: how new and exciting a suggestion looked to someone scrolling past it. It never checked Dulcie's actual list, no peanut, under a hundred and forty dollars, meals that repeat on purpose. The Regulars checks all four of those, every week, because that's the only way it gets anything right twice in a row.

At its worst, this is nine weeks of engineering time already spent on Surprise Basket, with seven more weeks already booked for a bigger version of it: more themes, more seasonal rotations, more surprise. Meanwhile The Regulars, the one feature that actually earned a place in people's weeks, never got a single one of those weeks, because nobody had a number that said it deserved them.

The lesson: a feature that wows someone once and a feature that earns a place in their week are answers to two different questions. A demo can only ever answer the first one.

Now here is the same thing as a story

Stage five above compresses this into four sentences. Here's the six weeks underneath it, the part a stand-up answer skips.

Dulcie Holmqvist can build a week of meals for her family in about ten minutes, standing at the kitchen counter with her phone before her Sunday shift starts. Six years of doing it by hand taught her the shortcuts: which dozen meals repeat well, what her son can't touch, and what a hundred and forty dollars actually buys at Larderwell's prices.

When The Regulars shipped, Dulcie turned it on in her first week and mostly forgot about it. It quietly reordered the same rotation she already trusted it with, milk, the peanut-free bread, the chicken thighs, and it kept getting the small stuff right: the week she ran out of coffee two days early, it caught it. She still built the exciting parts of her list by hand. The boring third of it just happened now.

Then, on a Tuesday, Surprise Basket showed up on her home screen. A dinner she'd never have picked herself: a harissa lamb kefta with a yogurt sauce, ingredients she'd never bought, styled in a photo good enough that she ordered it that same afternoon.

The product team decides what gets built next. Dulcie, on the other end of the app, has whatever they decided and nothing else.

It was genuinely great. She and her son ate well that night, and she told a friend about it the next morning.

The following Sunday, Surprise Basket suggested a peanut noodle bowl. Nothing about the recipe knew her son couldn't touch peanut at all. She spent four extra minutes checking every ingredient by hand before she'd trust it near her own kitchen, more time than her old list ever cost her, and skipped it. The Sunday after that, she didn't open Surprise Basket at all. She built her list the way she always had, and let The Regulars handle the rest.

Nobody at Larderwell noticed any of that happening. There was no complaint, no support ticket, nothing on a dashboard turning red. Six weeks after launch, Torgeir Draganescu, a data analyst on Sixten's team, pulled a routine cohort report, not because anything looked wrong, but because it was the report he ran every six weeks for a completely different feature. Surprise Basket had a column in it almost by accident.

Four of these five weeks looked completely fine, if the only thing anyone was watching was the trial number.

Of the sixty-one percent of shoppers who'd tried Surprise Basket in week one, six percent had opened it again by day fourteen. Torgeir read the number twice before he forwarded it to Sixten.

It was never really about the recipes being wrong. The recipes were fine, sometimes genuinely great, the harissa lamb kefta was real. The real cost was nine weeks of engineering time already spent, and a second version of Surprise Basket, more themes, more seasonal rotations, already seven weeks into its own roadmap slot, being built on top of a number nobody had checked.

The decision Sixten would take back sits in a meeting eight months before any of this, when Larderwell first wrote down what "a successful launch" meant. Week-one trial rate above forty percent counted as a win. It was a sensible bar at the time. Every feature the team had shipped up to that point was the kind of thing you either needed or you didn't, and trial rate and real use tended to move together. Nobody in that meeting had a reason yet to ask what happens when a feature is built to be exciting rather than useful.

So here's what changed. Run the same six weeks again, with the bar already written down: day-14 return has to clear twenty-five percent before Surprise Basket, or anything like it, gets called a win. Week one still lands the same, sixty-one percent try it, the demo still scores a nine. But by day fourteen, the number sits on Sixten's dashboard automatically now, not waiting on a data analyst's unrelated report. He sees the six percent in week three instead of week six, three weeks earlier, before the second version's roadmap slot ever gets approved.

One design hands the team a number that only ever tells them how good the first Tuesday was. The other hands them a number that tells them if there's a second one.

What I'd tell myself, sitting in that meeting eight months earlier: a bar that only measures the first try isn't a lower bar, it's a different question, and we'd been answering it for so long we forgot to notice when the question changed under us.

GUARD, weighed against one great Tuesday and the quiet cliff right after it

This was never really about whether Surprise Basket's recipes were good. They were. GUARD is for naming who pays when the number that gets celebrated isn't the number that turned out true.

Four of these five steps happened exactly as designed. The missing one, a required check on what happens after the first try, was never built into the path.

Only one axis matters for whether a feature needs the bar: is it meant to earn a place in someone's week, or is it meant to be a nice surprise exactly once.

The trade-off, said out loud: running every candidate dish through Dulcie's real constraint profile before ranking it for novelty adds about 300 milliseconds to building a basket, and it cuts the number of genuinely wild, never-tried-before suggestions Larderwell can offer each week from about five down to two. Larderwell took that trade on purpose, for a feature meant to earn a weekly habit, instead of shipping something fast and exciting that only ever works the first time.

And if you want to be sure it really works, try it somewhere else

Same five letters, a language app instead of a grocery app, and this time the ignored constraint is a learner's real mistake history instead of a peanut allergy.

Tonguecraft, built by Northfold Learning, teaches adults a new language through short daily practice. Wioletta Mirosava owns Tonguecraft's AI roadmap the way Sixten owns Larderwell's.

Same trap, a different subject. The branch that wins the demo is still the branch that loses the second week.

Roleplay Mode drops a learner into a spoken scene, ordering coffee in Lisbon, haggling at a market, with a theatrical AI voice playing the other part. Fifty-four percent of learners tried it in week one, and it was the single best-reviewed feature in Tonguecraft's history at launch. Daily Review is a short, unglamorous quiz that only ever tests the exact words and grammar rules a learner has personally gotten wrong before. Twenty-two percent tried it in week one, less than half of Roleplay Mode's number.

Same rank, mapped onto Tonguecraft: size the win by day-14 return, not the launch-week reaction. Roleplay Mode's day-14 return came back at five percent, because the scenes are dramatic but generic, they never draw on what a specific learner actually struggles with. Daily Review's day-14 return came back at sixty-one percent, because by definition it can only ever quiz you on your own real mistakes, so it keeps being useful in a way a fresh scene never has to be.

Swap the trigger and it still runs.
Speed: an interviewer caps you at ninety seconds. Skip straight to it: never let a launch-week number alone justify a bigger roadmap slot. Ask for the two-week return number before you approve anything.
Cost: there's no budget this quarter for a full cohort study. Pull a rough number from the last twenty triers by hand instead of zero. A small real sample beats trusting a demo score that was never designed to predict repeat use.
The model got better, for real: say Roleplay Mode's dialogue engine improves enough next quarter to actually reference a learner's real mistake history mid-scene. That's a reason to re-test the day-14 number and raise the bar it's held to, on purpose, with new data behind it, not a reason to have assumed it would happen from day one.

Where people run it wrong.
They treat a strong demo reaction as proof the feature works, instead of proof it's interesting the first time.
They let the launch-week number become the story everyone repeats at the next all-hands, even after a real cohort report says something different.
They wait for someone to notice the silence, a feature nobody talks about anymore, instead of gating any bigger investment behind a dated retention check.

How to use it live. Before answering, ask out loud: "does this feature win because it's new, or because it fits something the person actually needs every week?" Naming that split buys you a few seconds of thinking time, and it's most of the real answer.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0