In my experience, job applicants rarely ask the right questions when faced with a retention exercise. You can create a retention chart for anything, but I think these are the questions you simply have to ask:
- What do the cohorts measure?
- What is the criteria for retention (the % of the cohort that returns)?
- How can I segment the retention data?
There are many follow up questions you should be asking based on the answer.
Question 1: What do the cohorts measure?
- Is it any People? Companies? Revenue? People using it at a company? Is it a subset of any of those concepts?
- If it’s people — what kind of people? Is it people who did some action? Example: is it people who signed up for a product? Is it people who saw some value from your product? Bought a product? What is the trend of the cohorts over time? What could affect that? How do you expect the numbers to change over time?
Question 2: What is the criteria for retention?
- How is retention measured? If it’s people, what action do they need to take to be counted as having been retained? Why was that chosen? Why is it important?
- If it’s revenue, how is it measured?
- If it’s an attribute of a person or a company (maybe number of people active), how is that measured? Why was that chosen? How is it important to the business?
- Does it level off? If you have a budget to acquire people in the cohorts, how long must they stick around in order to sustain a profitable business?
- How does the data change over time? Are there patterns that you would expect to see, or cliffs where things drop off?
- Do you expect to see patterns horizontally (step function changes from one cohort to another), vertically (time barriers where cohorts worsen or improve), or diagonally (holidays, press spikes, re-engagement campaigns)?
Question 3: How can I segment the retention data?
- Are there pockets in the data that are better or worse than the overall average?
- Is there a feature that results in better retention?
- Is there a platform (ex: Android, iOS) that is better than another one?
- Is there a cohort acquisition source that performs better?
- Is there a geography that performs better in retention?
- Is there a cohort attribute that results in higher retention? Age / sex / profession of people? Size of company? Industry of company?
If you’re doing a retention exercise in an interview, you don’t even really need to look at the data, you could just ask questions the whole time.