How To Keep Your Users | Startup School

Y Combinator's famous motto is "make something people want." While simple to state, founders often struggle to answer a crucial quantitative question: Did we actually do it? The single best way to answer this question mathematically is by tracking cohort retention.

1. The Core Definition: What is Cohort Retention?

At its highest level, cohort retention is the practice of tracking what fraction of new users continue using your product over time. Instead of looking at your entire user base lumped together (which masks underlying churn), you isolate individual groups of new users who started at the same time and track their individual survival rates.

Note: David Lee (YF Group Partner and early co-founder of Bump / Google Photos) spent years giving hand-wavy answers to VCs about retention before learning the hard way that unflattened retention curves spell doom for a startup.

2. The Three Required Pillars of Cohort Analysis

Before building a cohort retention chart, you must precisely define three variables:

1. Cohorts (The Grouping): How you group new users. The most common baseline is by acquisition time (e.g., weekly or monthly cohorts: users acquired in January vs. February). Advanced slicing can include country, device, or acquisition channel.
2. Actions (The Active Definition): What specific user behavior qualifies them as "active" in a given time period. Avoid vague actions like merely opening an app; choose actions that correlate with receiving real value.
3. Time Periods (The Granularity): The time bucket used to measure subsequent usage, which must match your product's intended usage frequency.

Frequency Matching Table

3. Real-Life Analogy: The Party Visitor Log

Real-Life Analogy: Think of your startup as a perpetual party. Every time a new guest arrives, you hand them a colored sticker representing the month they arrived (e.g., a January sticker or a February sticker). At any given party night, you look around the room and count how many people are wearing January stickers versus February stickers. That is precisely how cohort tracking divides and monitors your user base over time.

4. How to Read Triangle Charts and Retention Curves

To visualize cohort data, founders build a Triangle Chart where rows represent acquisition months and columns represent subsequent months of activity. Each user is counted a maximum of one time per time period.

By normalizing these numbers against the initial cohort size, we get percentages, which can then be plotted on a Cohort Retention Curve (Percentage Retained on the Y-axis vs. Time Elapsed on the X-axis).

5. The Only Rule That Matters: Flat Curves

When evaluating retention curves, founders often fixate on the absolute height of the curve. However, the shape of the curve is infinitely more important.

• Product A (Downward Slope): If your curve continues sloping downward toward zero, you have a leaky bucket. You are trapped on a treadmill of constantly acquiring users only to lose them. You have not yet made something people want.
• Product B (Flattening Curve): Even if the curve flattens out at a seemingly modest number (e.g., 20%), a flat line means you are retaining a permanent core of loyal users who will accumulate over time.

Google Photos Example: Shortly after launch, Google Photos experienced weekly retention that dropped quickly before flattening between 20% and 40%. Though losing 80% initially sounds alarming, that flat 20% core gave the team absolute confidence that the product would scale to over a billion users.

6. How Founders Fool Themselves (Common Analytics Pitfalls)

Founders routinely delude themselves into thinking their retention is healthy by committing four major traps:

1. Picking Too Large of a Time Period: Widening your measurement window from weekly to quarterly guarantees more users will return within that window, artificially inflating and flattening your curves. Bump fell into this trap before VC meetings.
2. Picking Too Shallow of an Action: Counting any app open or notification click as "active usage" creates vanity metrics.
   • Example: Google+ once measured active usage based on whether users clicked a red notification bell in the corner of Gmail. The metrics looked massive, but cohort retention exposed it as an empty illusion.
3. Using "Is Paying" as the Active Action: Counterintuitively, customers stop using products long before they cancel their subscriptions (e.g., streaming services like Netflix). Always pair payment data with actual product usage.
4. Looking at a Single Point in Time: Celebrating a random data point (e.g., "We have 75% week-3 retention!") without inspecting the entire trend line. If week 2 was 90% and week 4 is 20%, your product is still plummeting toward zero.

The Golden Rule for Choosing an Action: Sit next to a customer using your product. When you watch them, what specific action makes you say, "Yes, they are using it exactly as intended"? Use that action for your retention curves.

7. Actionable Strategies to Improve Bad Retention

If your cohort curves are plunging toward zero, you must intervene. Four primary levers can rescue your product:

• Improve the Product: Speed up the app, reduce latency, simplify workflows, or discover entirely new sticky use cases. Look for subsequent cohorts to flatten out sooner and higher.
• Acquire Better Users: Bad retention is often a targeting error, not a product error.
  • Example: Google once ran a major marketing push targeting Gen Z for Google Photos. The campaigns brought in massive numbers of young users, but their retention was atrocious because young people do not yet have decades of life memories to reminisce over.
• Slice Your Cohorts: Break down your data by country, company size, device, or acquisition source. You will often discover that certain segments have completely flat, healthy retention while others drag the average down.
• Invest in First User Experience (Onboarding): Customers often fail because they don't know how to bridge the gap between their old workflow and your product. Streamlining onboarding is often the cheapest way to boost retention.
• Leverage Network Effects: Build dynamics where every new user makes the product inherently more valuable for existing users (e.g., communication apps, marketplaces).

8. The Holy Grail: The Layer Cake Chart

When you align your historical cohorts in absolute calendar time rather than relative time, you can stack them into a Layer Cake Chart.

When your total active user count grows over time not just from brand-new acquisitions, but because thick, accumulated layers of old cohorts refuse to leave, you have built a generational, multi-billion-dollar machine.

Summary and Key Takeaways

• Cohort Retention Defined: Tracking what fraction of specific new user groups continue using your product over subsequent time periods.
• The Three Pillars: You must isolate cohorts (usually by acquisition date), define a high-value action (correlated with real value), and choose a time period matching your product's natural cadence.
• The Shape Matters Most: Absolute percentage matters less than whether your retention curve eventually flattens out horizontally.
• Avoid the Treadmill: If your curves slope all the way to zero, you are trapped churning old users while burning energy acquiring new ones.
• Beware of Vanity Metrics: Don't fool yourself by widening time windows, tracking shallow notification clicks, or relying solely on subscription payment status.
• Build Your Own Charts: Build initial retention curves manually in Google Sheets or via raw logs to ensure your third-party analytics tools are measuring true cohort separation.
• Targeting Fixes: Poor retention is frequently a sign that you are marketing to the wrong customer demographic rather than a broken product.
• The Layer Cake: Ultimate startup success manifests as an expanding active user base powered by thick, accumulating layers of loyal users from past cohorts who refuse to churn.