System Design 14: Design Top K Leaderboard | Ranking System | Trending Songs | Movie | HLD | LLD

Designing a leaderboard or a trending list system (such as top $K$ songs on Spotify, top $K$ videos on YouTube, or global gaming leaderboards) requires serving low-latency requests from billions of records while handling massive write volumes. This chapter explores functional and non-functional requirements, core entities, REST/WebSocket API contracts, and three progressive architectural deep-dives ranging from Redis-backed caching to hybrid precomputed storage pipelines.

1. Problem Statement & Real-World Mental Model

The core objective of a leaderboard system is to efficiently compute and retrieve the top $K$ items (e.g., top 50 players, top 100 trending videos) out of billions of data points based on specific metrics like views, likes, or scores.

Real-Life Analogy

Mental Hook: Imagine a global digital stadium where millions of fans are casting votes or scoring points every single second. You cannot manually sort a billion scorecards every time someone asks for the current top 10. Instead, you need a high-speed scoreboard manager stationed at the entrance who instantly updates rankings on a sorted bulletin board, backed by an archive room that stores historical performance reports.

2. Functional Requirements

Before jumping into architecture, the functional requirements must be explicitly agreed upon:

• Data Ingestion: Users or client applications must be able to insert, update, or delete score/view/like events.
• Top $K$ Querying: Clients must be able to query the backend to retrieve the top $K$ trending items (songs, videos, or player scores).
• Time-Window Filtering: Queries must support specific time frames (e.g., past 1 hour, past 24 days, last 90 days, or all-time).
• Real-Time Updates: For interactive leaderboards (e.g., live gaming tournaments), the system must push near-real-time updates to clients.

3. Non-Functional Requirements & Scale Estimations

System design choices depend heavily on scale, latency constraints, and consistency models:

• Write Scale (Ingestion Load): Designed to handle massive write volumes—targeting 1,000,000 requests per second (e.g., global video views or song plays).
• Data Volume: Billions of items (songs, videos, or players) persisted in backend storage.
• CAP Theorem Trade-off: Optimized for High Availability (AP) rather than strict consistency. Returning slightly stale leaderboard data for a few seconds is acceptable; returning a 404 or service unavailability error is not.
• Latency SLAs:
  • Top $K$ Fetch Latency: $\le 100\text{ ms}$ (near real-time).
  • Data Update Latency: $\le 500\text{ ms}$ (soft real-time updates).
• Accuracy: Designed for accurate results rather than probabilistic approximations (though Count-Min Sketch or probabilistic structures can be discussed as advanced trade-offs).

4. Core Entities

Based on the requirements, three core data entities drive the system:

1. Event / Score Attribute: The metric driving the rank (e.g., views, likes, points, play-count).
2. Entity Record: The subject being ranked (e.g., video ID, song ID, player ID).
3. Time Frame / Window: The temporal dimension of the query (e.g., hourly, daily, 90-day, all-time).

5. API Design & Communication Protocols

To fulfill the functional requirements, the public API exposes clean REST endpoints and WebSocket channels:

Parameter Constraints:

• $K$ Upper Bound: Bounded between $1$ and $10,000$ records, managed via pagination to prevent payload bloat.
• Region Filtering: Essential because trending music in India differs significantly from the US.

6. High-Level Architecture (HLD)

At a high level, the system separates ingestion traffic from query traffic using a microservices pattern:

Component Breakdown:

• API Gateway & Load Balancer: Handles authentication, rate limiting, and distributes incoming traffic using a Round-Robin algorithm.
• Score Service: Receives high-throughput ingestion requests (views, likes, scores) and persists raw records.
• Ranking Service: Computes and retrieves top $K$ trending items for client requests.

7. Deep Dive: Architectural Evolution (3 Ways to Solve the Problem)

Because a single server cannot process 1M writes/sec or fetch top $K$ elements from billions of rows in real time, we evaluate three architectural approaches of increasing sophistication.

Approach 1: Redis Sorted Set (ZSet) In-Memory Approach

To solve top $K$ queries efficiently, we draw inspiration from DSA (e.g., LeetCode 347: Top K Frequent Elements) which uses heaps and priority queues. In system design, Redis Sorted Sets (ZSet) provide built-in in-memory sorted rankings.

Architecture Flow:

How it Works:

1. Incoming events are published to a Kafka Broker to buffer the massive write load.
2. A DB Consumer persists raw records into the primary Score Database.
3. A Redis Consumer updates a Redis Sorted Set (ZSet), where member keys are entity IDs and scores are the sorting metric. Redis automatically maintains sorted order.
4. The Ranking Service queries Redis directly to instantly pop top $K$ elements.

Pros & Cons of Approach 1:

• Pros: Super-fast data retrieval ($\le 100\text{ ms}$ latency); Redis automatically handles sorting without server-side compute overhead.
• Cons:
  • Single Point of Failure (SPOF): A single Redis instance holding billions of items will crash under memory pressure.
  • Time-Window Limitations: Storing all-time data in a single ZSet makes querying sliding windows (like past 30 days) extremely difficult.

Approach 2: Precomputed Storage with Apache Flink & Time-Series DB

To eliminate Redis memory limits and handle complex time windows, Approach 2 shifts toward batch and stream precomputation.

Architecture Flow:

How it Works:

1. Raw events are ingested and persisted in a write-optimized NoSQL database like Cassandra.
2. Apache Flink (or scheduled cron pipelines) continuously reads from Cassandra, aggregates scores across regions and time windows, and writes precomputed results into an Aggregated Database (such as InfluxDB or partitioned PostgreSQL).
3. The Ranking Service queries the Aggregated DB directly.

Pros & Cons of Approach 2:

• Pros: Highly scalable; handles complex historical time-window queries with ease; robust data durability.
• Cons: Not Real-Time. Due to stream processing lag and Flink aggregation intervals, top $K$ items are eventually consistent rather than instantly updated.

Approach 3: Hybrid Architecture (Redis ZSet + Aggregated DB Backup)

To combine the lightning-fast responsiveness of Approach 1 with the durability and scalability of Approach 2, we build a Hybrid Architecture.

Architecture Flow:

How it Works:

• Real-Time Data (Redis): Redis ZSets store current trending data with explicit Time-To-Live (TTL) values (e.g., retaining only the last 30 or 90 days).
• Historical Data (Aggregated DB): Past data and longer timeframes are precomputed and stored in the Aggregated Database.
• Fallback Logic: When a client requests rankings:
  1. The Ranking Service checks Redis first for immediate, real-time top $K$ items.
  2. If the request involves historical windows (e.g., "top songs from last January"), the query falls back to the Aggregated DB.
• Durability Backup: A background worker syncs Redis data to persistent storage so that if a Redis cluster node crashes, state can be recovered from snapshots without full recomputation.

8. Summary and Key Takeaways

• Problem Definition: Designing a Top $K$ leaderboard requires ingesting massive write volumes (up to 1M req/sec) while serving low-latency ($\le 100\text{ ms}$) read queries across billions of items.
• Functional Scope: Systems must support data insertion, top $K$ retrieval, time-window filtering (hours, days, all-time), and real-time updates.
• CAP Theorem Stance: Leaderboards prioritize Availability (AP) over strict consistency, favoring eventual consistency and fast response times over 404 errors.
• API Flexibility: Use REST endpoints for standard historical queries and WebSockets for live, interactive gaming leaderboards.
• Evolution of Solutions:
  • Naive/Simple Scale: Redis Sorted Sets (ZSet) provide in-memory sorting but suffer from memory limits and single-node failure risks.
  • Large Scale / Batch: Apache Flink combined with time-series databases handles massive datasets and complex windows, but introduces aggregation lag.
  • Production-Grade Hybrid: Combines Redis ZSets (with TTLs for current windows) and Aggregated Databases (for historical data and fault recovery) to achieve both real-time responsiveness and system durability.