7 sins of user retention

"Avarice" by Pieter van der Heyden

In 2023, market conditions are pushing the shrewdest founders to focus on one metric above all else: retention. Everyone knows retention is important to startup success, but this becomes especially critical when capital is scarcer. You can no longer afford to grow by acquiring users that don’t stick.

‍Why not? Well first, your cost of capital has gone up…by a lot. That venture money for buying those ads and hiring salespeople, is more expensive, i.e. dilutive, to you now. Second, CAC is higher because conversion rates are lower. When there’s less money available and the money you do have doesn’t get you as many new users as it used to, you need a different strategy.

Even those clever founders who have battened down the hatches to focus on retention this year have won only half the battle, because retention is a tricky, nuanced topic. I screwed this up badly as a founder. Made every mistake in the book. I see those same mistakes repeated by founders far cleverer than me, year after year. So here are 7 common mistakes and pitfalls that even experienced founders make, but that can be deadly during these times of pricey capital.

1. Counting “dormant” users as retained

This problem occurs when a subscriber buys an annual plan, stops using the product after 2 weeks never to be seen again, and is counted as “retained” for the next 11 months because technically their subscription is still active and the credit card billing hasn’t been shut off yet. Here, you’re looking at payment retention.

Payment retention when a user has stopped using the product is effectively the ugly cousin of breakage, like unused gift cards. If we’re trying to assess our degree of product-market fit, we want to count a user as retained based on their usage. The user needs to get some value out of the product to count.

Depending on what you’re trying to achieve, ask yourself whether you actually care about payment retention or usage retention.

2. Treating all your users as one homogenous group

Blending all of your users into one big fat cohort might not tell you much. Your data will likely be too noisy. You’ll mix some users that fit perfectly into your ICP and retain forever, with others that should never have been onboarded to begin with. Segment your retention so you can see how different types of users retain differently. Slice many different ways: geography, industry, company size, age, funding status, tech stack, features they adopted, etc. Drilling down will teach you about your ICP and how to increase retention in narrower groups. Eventually, these insights will help you allocate marketing dollars more effectively too.

"A sloth churning butter, old photograph" by Dalle-2

3. Failing to quantify churn reasons

You can’t improve your churn metric if you don’t deeply understand which reason wreaks the most havoc. Most founders I meet have a high-level grasp of the top churn 2-3 reasons but not which is the biggest and not by how much. If you want to improve your churn, get scientific. Get the numbers. Attack the area you believe will have the biggest impact. No hand-waving.

You might break down different types of churn, and within each type, reasons. A few examples:

Further reading: My partner David (somewhat famously) wrote that the top 2 reasons for SaaS churn are #1: poor onboarding, and #2: loss of champion, i.e. your sponsor at the buyer company quits.

4. Benchmarking randomly

This is a lengthy one… Founders looking at their retention rate usually want to know whether it’s “good” in some objective sense. So we try to benchmark and find comps. Several blog posts out there give examples of retention rates of popular products. But 2 problems with benchmarking should make us skeptical:

(a) Companies measure it in wildly different ways.

There is no GAAP for retention. This matters, as a small difference to the mathematical formula produces a large difference in the outputted numbers. You can’t really trust the metric without knowing how it’s measured. How do you define an active user? How do you count pilots or trials? Resurrections? Upsells / downgrades? Payment vs. usage retention? Delinquencies? Non-regrettable churn (i.e. you fired the customer)? It’s hard to benchmark yourself against a peer set when the numbers are not computed in a standardized way.

(b) All products aren’t built to be used for the same duration of time.

A product that has 80% 12-month retention does not necessarily have more product-market fit than one that has 40% 12-month retention. You need to look at the user’s “job-to-be-done” and whether it’s being achieved.

The job-to-be-done is the the reason your customers find you. Your job-to-be-done statement should follow the words: “I want to…” As in, “I want to buy a car” for an online used car marketplace, for instance.

Successfully achieving your customer’s job-to-be-done is the thing that delights them and makes them recommend you to their friends. A used car marketplace can delight a customer in 3 weeks, as they successfully buy their car and move on. That’s how I bought my car. Despite my “churning” after 3 weeks, I go onto tell everyone how well the product works and how delighted I am.

By contrast, an online college whose job-to-be-done is to earn its customers a bachelor’s degree may need to retain customers for 4 years. Anyone dropping out after 3 weeks probably doesn’t feel the product succeeded for them. So, just be careful not to compare apples and oranges.

5. Flipping to growth mode too quickly

Much discussed by others before me… The anti-pattern here typically stems from a combination of pressure from VCs, a subconscious feeling that you need to spend the money you raised, and a mistaken view that you need to “make up for” poor retention by spending more on marketing and sales so you don’t miss your growth target. Put all that energy and money into fixing the product to the point it’s consistently achieving the job-to-be-done.

"The fall of Icarus" by Jacob Peter Gowy

6. Flipping to growth mode too slowly

Everyone knows about prematurely scaling up acquisition, but the exact opposite doesn’t get talked about as often. I’ve seen founders make “scrappiness” such a core part of their own – and by extension their company’s – DNA that it becomes difficult for them to flip their mindset to growth even after their product is retaining customers. A scalable acquisition engine is a very different muscle your company must build in order to become valuable. New technology opportunities tend to exist for a finite period of time.

Everyone hears about the big flameouts where overzealous teams flew too close to the sun, spent all their money, and ran the plane into the ground. Makes for sensational Techcrunch articles. You rarely hear of the stable, slow-growth company that got lapped either by the next technology step-change that made their product irrelevant, or by a more aggressive team that swooped up the market. The “conservative” startup becomes a zombie company, and those generally don’t make for very sensational news stories.

7. Taking bad retention personally

Product-oriented founders seem to take retention metrics personally. Often, a founder’s entire identity is wrapped up in their product. You present a compelling story, only to have your darn retention numbers ruin that narrative. This results in becoming defensive during the period before the product really starts working. Most product-y founders would rather admit that they haven’t figured out go-to-market than that their users are abandoning them in droves. I’ve heard stories of famous founder/CEOs fudge numbers and effectively lie to their own team about retention because the truth somehow feels shameful. Remember: it’s rare to build something with great retention on the first try. Iterate. Be intellectually honest about it, especially with yourself. Can't fix things if you’re not.