The retention data behind habit-building products

Disclosure: Mixpanel, the Data Analytics company, has previously sponsored Hacker Noon.

Data-driven product teams care about more than page views, unique visitors, and account sign-ups. They want to know how many of those users become customers and how many of them remain paying customers — for a month, a year, or a lifetime. Then, they want to know how to drive those numbers up.

We interviewed two of Mixpanel’s retention experts. In this article, Tim Trefren, Mixpanel co-founder and Head of Customer Success, and Jenny Li, one of our lead product managers, share how they think about retention and the strategies that have worked for them in the past.

But our team knows better than anyone that it can be hard for a team to set goals for customer retention when they don’t even know what good retention looks like. In competitive markets filled with fickle consumers, how many loyal customers can even industry giants keep, year after year? And what percentage of those customers fall off along the way?

These questions get tossed our way often, so we answered them in our 2017 Product Benchmarks Report. We aggregated behavior data from 1.3 billion unique users who triggered more than 50 billion web and mobile events across hundreds of products run by Mixpanel customers. Considered together, our customer’s data revealed — among other stats — what median and best-in-class retention is for software-as-a-service, finance, e-commerce, and media & entertainment.

Of course, how companies define what counts as a retained user for them is shaped by their business goals and what behaviors they want to see their users taking. Similarly, the time frame that teams use to measure retention — whether daily, weekly, monthly, or yearly — depends on their product, their usage, and their goals. In this report, we defined retention as broadly as possible: the rate at which users who had already performed a single action came back and performed another. We calculated those rates on a weekly basis, meaning that individuals who used a product on Week 1 and came back sometime in the next week would count as a retained user.

Regardless of the specific definition or time frame, retention is an important metric for understanding the health of the business. For product teams in particular, it helps them see if they’ve built something that’s valuable to their customers in the long-term.

As shown in the graph below, median products tended to retain around 30% of new and existing users after the first week. That number dropped down to the low teens by Week 8. The outlier is e-commerce, which tends to retain half as many users as the other three industries.

Products in the 90th percentile had significantly higher retention after Week 1, ranging from 63% and 74%. By Week 8, that number dropped to between 25% to 38%, still a higher percent of users than the median products had after one week. In this case, top performers in e-commerce retained a higher percentage of users than other industries. While 90th percentile retention for SaaS, finance, and media & entertainment products was 2x that of median products in those industries, e-commerce had a 5x difference between median and 90th percentile. We explore this split in the report.

Clearly, the gap between median and 90th percentile is vast. Certainly, a company’s business model will affect what they should expect for retention. But companies who are not performing as well as their competitors need to develop strategies to move retention — whether from the bottom to the median or from the median to the 90th. To that end, we interviewed Tim and Jenny to find out how they think about retention. Here are the three takeaways:

When defining retention for your product, think about what it will take for you user to develop a habit.

B2B companies and B2C companies have to measure retention differently. For example, at Mixpanel, a B2B company, our product teams think more about weekly and monthly retention. As Tim notes, just because a user doesn’t go to their Mixpanel account every day doesn’t mean they won’t use it next week. Users could develop a weekly or even monthly habit of checking the data in Mixpanel, and they would still count as retained.

On the flip side, for some B2C companies, it might make more sense to define retained users exclusively as those who use the product every week or even every day. Gaming and social media product teams tend to see more success with users who go on their apps every day, so they define their retained users accordingly.

Regardless of how product teams define what counts as retained users, they should always pay attention to the behavior of very active users and high value customers. According to Jenny, this research can uncover to the behaviors that tend to be habit forming, and help them understand what kinds of actions they should encourage with their onboarding flows and customer marketing campaigns. Study and replicate those flows.

Retention is a funnel, so focus on early retention first.

It’s much easier to get a new user to develop a habit than it is to re-engage lapsed users, says Jenny. That means product teams at B2B and B2C companies alike should direct more of their energy and resources toward increasing early retention.

Moreover, the distinctions between the users who never come back and the ones who remain at Month 12 can typically be traced back to the differences in retention in the first month. Many B2B companies who have annual plans tend to look at their month twelve retention. But even those teams should pay attention to user behavior in the first month because the problems with Month 12 retention are likely just an amplification of what went wrong in Month 1.

Tim shares what that looked like at Mixpanel: “We found that users who were invited to join an existing project have a significantly better retention rate at Month 6 and Month 12 than the users who signed up on their own. This intuitively makes sense. Users who were invited by co-workers to join an existing a project already had data to explore. That means they didn’t have to do any set-up to start seeing value. But users who signed up on their own and stuck it all the way through Month 1 tend to behave just like invited users by the Month 12 mark.” Our team wouldn’t have found that pattern in retention if they had focused only on Month 12.

High retention rates don’t always mean success.

Companies with median or even 90th percentile retention rates still shouldn’t grow complacent. According to Jenny, products that have high retention rates could have other problems happening under the hood. If a product stops attracting new users, for example, retention rates will likely increase because the overall user base is going down and the survivors are artificially propping up retention numbers. In fact, there can be an inverse correlation between retention and how many users a product acquires in that time period, so product teams should always consider retention rates in the context of other metrics.

Again, to develop an accurate picture of how a product is performing, teams should have an understanding of what user journey makes the most sense for a given product and business model and then define and measure retention accordingly.

The only way to get started addressing a retention problem, however, is to diagnose it. Read the full Product Benchmarks Report to see how your product measures up.