WHATSAPP System Design: Chat Messaging Systems for Interviews

Designing a globally scalable, real-time chat application like WhatsApp is a cornerstone exercise for system design interviews. A robust chat architecture must support massive concurrent connections, lightning-fast message delivery, read/delivery receipts, user online/last-seen status, and efficient group messaging while maintaining strict memory and network efficiency.

1. Feature Scope and Interview Strategy

When designing a chat application in a 45-minute interview, you must aggressively triage features. Trying to build every bell and whistle leads to analysis paralysis.

Feature Categorization

• Core Features (Must Have):
  • 1-to-1 Chat: Reliable, real-time message exchange between two individuals.
  • Group Messaging: Multi-party messaging capped at reasonable size limits.
  • Sent, Delivered, and Read Receipts: Visual ticks indicating message state.
  • Online / Last Seen Status: Real-time presence indicators.
• Advanced / Secondary Features (Scope Out or Delegate):
  • Media Sharing (Images/Videos): Handled via object storage pipelines (delegated to standard architectural patterns like Tinder's media store).
  • Ephemeral vs. Permanent Storage: Privacy settings and compliance logs.
  • Authentication & Load Balancing: Standard platform services decoupled from chat logic.

2. Core Mental Model: The Gateway and Dumb Connections

To handle millions of concurrent persistent connections, we must structure our edge architecture carefully.

Real-Life Analogy: The Telephone Switchboard

Think of WhatsApp's gateway servers as a massive telephone switchboard operator whose only job is to plug callers into lines. The operator does not inspect, parse, or evaluate the deep meaning of every conversation; they simply maintain the physical wire connection and route the signal onward.

Why Dumb TCP Connections?

If gateway servers stored mapping information (e.g., "User A is on Box 1, User B is on Box 2") directly in their own local memory alongside active TCP sockets:

1. Memory Starvation: Maintaining millions of TCP sockets consumes heavy RAM. Storing routing metadata locally causes massive memory bloat.
2. Tight Coupling: If a gateway restarts or scales up/down, local routing tables are lost or become stale, creating systemic fragility.

Instead, we use dumb gateways that only pass raw bytes, offloading all session and routing state to a dedicated, distributed microservice.

3. Protocol Selection: WebSockets vs. HTTP

Traditional HTTP operates on a request-response model (Client requests, Server responds). This fails for chat applications where servers must push incoming messages to clients instantly.

4. 1-to-1 Chat Flow and Delivery Receipts

Let's trace how User A sends a message to User B, and how delivery receipts flow back.

Step-by-Step Execution Workflow

1. User A sends a message via WebSocket to Gateway 1.
2. Gateway 1 acts as a dumb pipe, forwarding the raw payload to the Session Service.
3. The Session Service queries its distributed database/cache to locate User B's active connection box (Gateway 2).
4. The Session Service triggers a parallel write to a persistent chat database/queue to ensure durability.
5. Gateway 2 pushes the message over the active WebSocket to User B.
6. Once User B receives the payload, an ACK flows back through Gateway 2 -> Session Service -> Gateway 1, triggering the Delivered receipt on User A's screen.

5. User Presence: Online and "Last Seen" Tracking

Tracking millions of users' online states requires distinguishing between genuine user interactions and background application polling.

The Problem with Naive Polling

If the server constantly pings User A or recalculates last-seen timestamps on every background delivery receipt, the database experiences heavy write amplification and performance degradation.

Architectural Solution

• Client-Side Smart Flags: The client application explicitly flags outgoing requests:
  • isUserActivity = true: User typed a message, opened a chat, or interacted with the app.
  • isUserActivity = false: Background sync, delivery ACK, or app polling.
• Last Seen Microservice: Only requests marked with isUserActivity = true hit the Last Seen service to update the user's timestamp key-value store.
• Presence Threshold Rules: If User A performed an activity within the last 10–15 seconds, the system displays an "Online" badge rather than an exact timestamp.

6. Group Messaging Architecture

WhatsApp limits groups to 200 members. Why? Because of the fan-out write amplification problem.

Fan-Out Strategy vs. Limits

• If a group had 1,000,000 members (like an Instagram broadcast), sending a single message would require fanning out 1,000,000 individual network writes, crashing gateways.
• By capping groups at 200 members, the system can reliably fan out messages via WebSockets without overwhelming the infrastructure or forcing resource-heavy pull architectures.

Group Message Routing Flow

1. Sender transmits a group message to its connected gateway.
2. The Session Service queries the Group Service to retrieve all member User IDs for that Group ID.
3. Using Consistent Hashing on the Group ID, the system routes the message efficiently to the respective gateways where group members are currently connected.
4. Message Queues (e.g., Kafka/RabbitMQ) handle retries asynchronously if a member's gateway is momentarily unresponsive.

7. Engineering Optimizations and Production Resilience

To maintain high availability during peak traffic spikes (e.g., New Year's Eve, festivals like Diwali), chat engineering teams employ several critical optimization techniques:

1. Parser Microservice Offloading

Gateways should never waste precious CPU cycles parsing JSON objects or authenticating headers.

• Pattern: Gateways forward raw byte streams to a dedicated Parser Microservice.
• Internal Protocol: Convert raw payloads into binary serialization formats like Apache Thrift or Protocol Buffers for high-speed internal inter-service communication.

2. Graceful Degradation & Deprioritization

When system load spikes beyond capacity limits:

• Drop Non-Essential Features: Temporarily disable "Last Seen" timestamp updates and "Read Receipts" (--read ticks).
• Prioritize Core Messaging: Allocate 100% of CPU and queue bandwidth to raw message delivery (send and deliver), ensuring the core chat utility remains operational under heavy strain.

3. Idempotency & Retries

Because network failures cause retries, message duplicate delivery is inevitable.

• Every client message must include a client-generated UUID (Idempotency Key).
• Downstream storage layers use this UUID to ensure that if a message is retried three times due to a dropped TCP ACK, it is only persisted and sent once.

8. Summary and Key Takeaways

• Dumb Gateways: Keep edge gateway servers light by offloading session state and user-to-box mappings to a dedicated Session Microservice.
• WebSockets over HTTP: Use full-duplex WebSockets for real-time bidirectional message delivery, bypassing HTTP request-response limitations.
• Explicit Presence Flagging: Differentiate between user-initiated actions and background app syncs using isUserActivity flags to maintain accurate "Last Seen" data without database thrashing.
• Controlled Fan-Out Limits: Cap group memberships (e.g., 200 users) to prevent destructive fan-out write amplification during group message broadcasts.
• Consistent Hashing: Use consistent hashing mapped on Group IDs to distribute load evenly across routing microservices.
• Protocol Buffers & Thrift: Use binary serialization for internal microservice communication to minimize network overhead.
• Graceful Degradation: During massive traffic surges (e.g., New Year's Eve), deprioritize non-essential telemetry like read receipts and last-seen updates to preserve core messaging throughput.
• Idempotency: Implement client-generated UUIDs across all messages to safely handle network retries without duplicating messages in storage.