Segment Your User Base: Depth of Engagement

If you haven’t read Jonathan Hsu’s 8 part Medium series on Social Capital’s diligence process, add it to your reading list. I didn’t immediately grok all of the concepts in the post, but it has had an incredible impact on how I look at product metrics.

It appears it’s a big part of their recent announcement of how they are able to fund early-stage companies focused exclusively on their metrics.

One of the concepts that struck me was the depth of engagement. It shows you how engaged different portions of your user base are. You don’t need a ton of fancy data science techniques to get a glimpse into what your user base is doing. All you need a fairly straightforward SQL query to get you started.

It starts with a fairly simple concept: how many users are active for 1 day in the past month? How many are active for 2 days in the month? It’s really simple to generate a histogram (this is fake data) that looks like this:

Count of Users by Days Active

In this fake example there are 100k monthly active users (MAUs) in this hypothetical product. I think this is very telling and interesting from a strategy and operational perspective, but there’s a different view that I now prefer. I prefer to look at this chart on a percentage basis (the % of MAUs), and look at it cumulatively. This is what it looks like:

CDF of Monthly Active Users by Days Used

How to read this chart: 33% of the MAUs are active for a single day of the month. It may be the first day of the month or the last, but the people that fall into this bucket were only active for a single day in the month. 53% of the MAUs were active for 2 or fewer days – you add up the 33k and the 20k from the histogram to get the 53%. In Jonathan’s example there’s a little bit of a spike of users that are active every day of the month – in a bunch of the examples I’ve seen in the B2B space there’s a nice healthy bump around 20 days, which makes sense when you consider that B2B apps are most likely used every business day, rather than every day.

This is a powerful way to slice up your install base very quickly. I push for taking the MAU install base and slicing it up into types of users. Here’s a hypothetical set of groupings:

  • Low engaged users (66%): 3 days of activity or less
  • Medium engaged users (14%): 4-10 days of activity
  • Highly engaged users (10%): 11+ days of activity

There are a bunch of plays that I could see happening for each of these buckets:

  • Sales: I could see sales following up with customers that fall into the highly engaged bucket. If they’re free, I could see them seeing value in paid tiers of your product. If they’re already paid customers, they are probably the most likely bucket to see value in additional paid options.
  • Services: I could see customer success reaching out to the low engaged bucket to understand why they aren’t using the tool more frequently. In a B2B company where customer success is focused on retention, this is an area of high potential churn.
  • Product: I could see the product team looking to build features that address the missing functionality users need to use it more. They could also work on retention hooks that pull users back into the product / get them to see more value in the tool.
  • Marketing: I could see the marketing group targeting users based on the bucket they fall into and how they might see value from additional features.

If you’re interested in doing this yourself, check out this Jupyter notebook for sample code.

2 Comments

  1. Great write up, Dan! The Jupyter notebook is especially helpful.

    What other user segmentation approaches do you most commonly use?

    • danwolch

      January 8, 2018 at 7:49 pm

      Since I’m in B2B, I like to look at MAU on an organization level.

      I like segmenting on a bunch of other criteria:

        Geo
        Industry
        SKU
        Free vs. Paid
        Language
        MAU

      I try to keep it simple to start since you can spend a lot of time going down rabbit holes.

Leave a Reply

Your email address will not be published.

*

© 2018 Dan Wolchonok

Theme by Anders NorénUp ↑