What is a data flywheel and what are its preconditions?

Corvid Bank's budgeting app is a single swipe on a phone, checked three or four times a day. Tobias Renke leads the categorization feature there, the one that sorts every purchase into "Groceries" or "Dining" or "Transport" automatically. He built the correction flow that was supposed to make the whole thing get smarter over time.

How to answer this, stage by stage

Nobody is scoring whether you can define "flywheel." They're scoring whether you can name the exact thing that breaks it, out loud, before it happens.

Let's learn

Here is what a data flywheel actually needs, and what a redesign can quietly take away from it.

Corvid Bank's budgeting app auto-sorts every purchase into a category, and if it gets one wrong, a user can fix it with a single tap. That correction was supposed to retrain the categorizer, so the whole system got smarter with every mistake anyone caught. Before a redesign changed the correction flow, 38 percent of miscategorized transactions got corrected within a week, regardless of whether the purchase was five dollars or five hundred.

The five letters, held up as one page. Reversibility is the step that separates a real precondition from a nice-to-have.

Then a redesign added a detail modal to the correction flow, asking for merchant type and whether the purchase was recurring, to feed a future "smart budgets" feature. What used to be one tap became three, and one of those taps meant typing.

Here's the turn: the extra taps themselves weren't the real problem. The real problem is what people do when giving feedback gets three times as expensive. They don't stop giving feedback altogether. They start rationing it toward the corrections that feel worth the extra effort, and quietly stop bothering with the ones that don't.

At its worst, the categorizer keeps getting sharper about rare, big-ticket purchases while it quietly forgets how to handle the recurring five-dollar coffees and eleven-dollar subscriptions that make up most of what people actually buy, which is exactly the part of the product a budgeting app is supposed to be good at.

What I would leave alone: I wouldn't add friction anywhere near the correction for genuinely one-off, high-value purchases, since people already correct those reliably and a detail prompt there barely costs them anything they weren't already willing to spend.

The lesson: a data flywheel isn't powered by how much people use the product. It's powered by how cheap it stays to tell the product it's wrong, for every kind of case, not just the ones people happen to care about.

Now here is the same thing as a story

The short version above is what you'd say defending the correction flow's design in a roadmap review. Read this one for how the skew built up over six months with nobody deciding it on purpose.

Every night before bed, half of Corvid Bank's users open the app for thirty seconds, scroll the day's spending, and tap to fix whatever got mistagged. It's a small habit, barely worth noticing, and it was the entire reason the categorizer kept improving.

One extra step doesn't sound like much, until it's the step standing between a cheap fix and no fix at all.

Tobias shipped the detail modal in March, to power a "smart budgets by merchant" feature planned for later that year. The team tested it, watched correction volume, and it looked fine. People were still tapping to fix things. Nobody split the number by transaction size, so nobody saw what was actually happening underneath the average.

Nobody decided, on any single day, to stop teaching the model about small purchases. The habit just quietly thinned out, one skipped modal at a time.

By month four, a five-dollar coffee mistagged as "Entertainment" instead of "Dining" sat uncorrected for weeks, because who's going to spend fifteen seconds filling out a merchant-type form to fix five dollars. A three-hundred-dollar mistagged furniture purchase still got fixed the same day, because at that size, the extra taps felt worth it.

Two clusters, not one blended average. Small, recurring spending fell into the corner nobody was watching.

By month six, Tobias pulled the categorizer's accuracy by segment for an unrelated review and noticed something odd: accuracy on recurring subscriptions had actually dropped four points over the same period the model's overall accuracy metric looked completely flat.

Corvid Bank's loop wasn't a graveyard and it wasn't starved for volume. It landed in the third branch: technically alive, quietly biased.

When the modal was proposed, someone in the room said, "let's just capture merchant type while we have the user's attention," and it sounded completely reasonable, since the future feature genuinely needed that data and asking felt like the obvious moment to do it.

Rerun the same six months with the quick one-tap fix kept intact, and the detail modal offered only as an optional follow-up for people who want to add it: small-transaction corrections stay near 35 percent instead of collapsing to 6 percent, subscription-category accuracy holds steady instead of slipping four points, and the "smart budgets" feature gets its merchant-type data from the smaller group of users willing to fill in the extra detail, without taxing everyone else's easy fixes to get it.

What I'd tell myself, watching six months of aggregate accuracy hide a real, growing hole: the modal was never wrong to want. It was wrong to make everyone pay for it, including the people fixing a five-dollar coffee who never needed to answer a question about merchant type at all.

ORDER, the ranking that would have caught this in month twoNot a definition exercise. ORDER is what tells you which precondition is the one you can't add back later.

The recap, one line per letter: outcome is a categorizer that keeps improving on every kind of spending, reversibility is that cheap universal feedback capture can't be added back after the fact, dependency is that capture has to exist before volume matters and volume before a pipeline is worth building, evidence is that a one-week segment split would have shown the collapse four months early, and rank puts cheap capture first, a real pipeline second, and segment monitoring third.

And if you want to be sure it really works, try it somewhere elseSame five letters, a precision-farming app instead of a banking app. Different flip family entirely, the same missing precondition.

Anselm Rurik runs product at Hallowfield Agritech, where farmers photograph a crop leaf and the app diagnoses disease, with a correction option if the app got it wrong. Mapped onto ORDER: outcome is a diagnosis model that keeps improving across every regional pest strain, not just the common ones photographed most. Reversibility says regional photo representation, captured from the start, can't be retrofitted; a "why this diagnosis" explainer screen can be added anytime. Dependency says you need enough regional volume before a local specialization feature is worth building. Evidence favors checking, cheaply, whether a given region's pest strains are even represented in the current training set before promising local accuracy gains.

The flip here is verification, not substitution. For one full growing season, the app's diagnoses were accurate enough that farmers stopped spot-checking them against their own field experience. Then a new pest strain arrived in one region, and the app confidently misdiagnosed it as a known, treatable disease for several farmers in a row, since that region's strain had never been in the training data at all. Once word spread between neighboring farms, farmers started manually re-verifying every single diagnosis against a physical field guide before treating anything, which undid the entire point of a fast, automated first read.

The third step is where Hallowfield's loop actually broke: corrections on the new strain were captured, but nothing fed them back before the damage spread between farms.

Swap the trigger and it still runs.
Speed: an interviewer caps you at sixty seconds. Say "a flywheel needs cheap capture, enough volume, and a pipeline that actually uses the feedback, in that order," and stop.
Cost: no budget to build a full bias-monitoring dashboard before the next feature ships. Say so honestly, and manually split correction rate by segment once a month until there's budget for more.
The model got better, for real: Corvid Bank's overall accuracy metric genuinely held steady the whole six months. That's exactly why nobody caught the skew. A healthy average is not proof every segment underneath it is healthy too.

Where people run it wrong.
They treat "we added a feedback button" as proof the flywheel exists, without checking whether giving that feedback stayed cheap for every kind of case.
They watch an aggregate accuracy number and assume it represents every segment equally.
They add a data-collection step to serve a future feature, without asking what it costs the everyday corrections happening right now.

How to use it live. The moment an interviewer asks about preconditions, ask yourself: which one of these, if missing today, could I never get back later? Answer that first, rank the rest under it, and the response builds itself from there.

Flashcards (tap any card to flip it)

Check yourself Score: 0 / 0