System Design Prep: Fundamentals, Frameworks, and Core Architectures

System design interviews evaluate an engineer's ability to architect scalable, resilient, and maintainable distributed systems under ambiguous constraints. Success requires more than memorizing technical components like caches, load balancers, or databases; it demands a structured methodology to navigate trade-offs, clarify hidden requirements, and justify design decisions under time pressure. This guide outlines the foundational framework of system design and applies it step-by-step across four classical system design benchmarks: a URL Shortener, a Distributed Rate Limiter, an Enterprise Notification System, and a Scalable Social Media Newsfeed.

1. The Philosophy of System Design Interviews

System design interviews simulate real-world technical decision-making under high ambiguity. In a typical 45-minute session, candidates are given intentionally vague prompts such as "Design Instagram" or "Design Dropbox." Interviewers do not expect a flawless, production-ready blueprint. Instead, they assess your engineering judgment, communication, and systematic thinking.

What Interviewers Actually Evaluate

1. Judgment Under Ambiguity: Can you transform a loose request into concrete, measurable engineering requirements?
2. Technical Depth: When proposing a component (e.g., placing a Redis cache in front of PostgreSQL), can you articulate both the specific problem it solves and the secondary complexities it introduces (e.g., cache invalidation, staleness, memory limits)?
3. Communication: Do you proactively explain your thought process and maintain control of the design narrative?
4. Collaboration & Adaptability: Do you validate key assumptions with the interviewer and incorporate feedback rather than defensively adhering to a flawed initial diagram?

Critical Anti-Patterns to Avoid

• Premature Architecture: Grabbing the marker or diagramming tool immediately to draw application servers, databases, and message queues before clarifying business goals and scale.
• Over-Engineering: Introducing multi-region deployments, sharding, distributed message queues, and multi-layered caching strategies for low-throughput applications (e.g., designing a massive distributed infrastructure for a system with 10,000 total users).
• Silent Designing: Working in isolation for extended periods without verbalizing trade-offs or checking alignment with the interviewer.

2. The Universal 6-Step System Design Framework

To maintain structure under interview pressure, apply a standardized 6-Step Framework. This roadmap prevents missing critical edge cases and balances time effectively across the 45-minute window.

Step 1: Clarify Requirements (~5 Minutes)

Convert vague goals into explicit Functional and Non-Functional requirements.

• Functional Requirements: What business features must the system support directly? (e.g., "Users can post photos," "Users can follow other users").
• Non-Functional Requirements (NFRs): How well must the system perform? Target latency thresholds, availability guarantees (e.g., 99.99%), data consistency requirements (eventual vs. strong), and retention policies.

Step 2: Back-of-the-Envelope Estimation (~5 Minutes)

Calculate the system's operational scale to dictate structural choices:

• Estimate Daily Active Users (DAU), Read-to-Write ratios, network bandwidth, storage requirements over a 5-to-10-year horizon, and Peak Queries Per Second (QPS).
• Determine whether the workload is Read-Heavy (demanding heavy caching and replication) or Write-Heavy (demanding message queues and append-friendly databases).

Step 3: Define the API and Data Model (~5 Minutes)

Establish the system's interface contracts and core storage schema:

• Draft clean RESTful HTTP endpoints, gRPC interfaces, or GraphQL schemas.
• Outline the core database entities, foreign key relationships, and data shapes.
• Select the storage paradigm (SQL vs. NoSQL vs. Object Storage vs. Key-Value) based on access patterns rather than buzzwords.

Step 4: High-Level Design (~10–15 Minutes)

Construct an end-to-end architecture satisfying the baseline functional requirements:

• Draw client interfaces, entry points (API Gateways / Load Balancers), core application services, storage tiers, and inter-service connection arrows.
• Walk through a complete request flow to verify that the design fulfills all basic user journeys.

Step 5: Deep Dives (~10–15 Minutes)

Focus intensely on the hardest architectural challenges specific to the problem:

• Zoom in on bottlenecks (e.g., handling celebrity fan-outs, mitigating hotkeys, ensuring atomic rate-limiting counters, preventing duplicate notifications).
• Discuss explicit trade-offs and evaluate alternative strategies.

Step 6: Bottlenecks and Scaling (~5 Minutes)

Perform a critical self-audit of the proposed architecture:

• Identify single points of failure (SPOFs), component failure recovery steps, database replication lag, and cache eviction side effects.
• Conclude with a concise 30-to-60-second summary of the final system design.

3. Requirement Gathering & Qualitative Trade-Offs

Asking the right clarifying questions sets the trajectory of the entire interview. Do not bombard the interviewer with generic, memorized lists (e.g., "Does this need to be scalable?" or "Should I use SQL or NoSQL?").

Rule of Thumb: Only ask questions whose answers directly alter your architectural choices. If an answer will not change your choice of database, API format, or caching layer, do not ask it. State a reasonable assumption and move forward.

Evaluating Question Quality

Guiding Questions for Non-Functional Requirements (NFRs)

1. Scale & Growth: What is the current DAU, and what is the expected growth rate over the next 12–24 months?
2. Access Patterns: What is the ratio of read operations to write operations?
3. Data Durability: What data can never be lost under hardware crash scenarios (e.g., financial ledger entries vs. temporary analytical logs)?
4. Latency Budget: What are the strict $p_{99}$ latency targets for key user-facing actions (e.g., checkout vs. background email generation)?

4. URL Shortener (bit.ly) — Read-Heavy Architecture

A URL Shortener accepts a long target URL (e.g., https://example.com/products/items/index.html?ref=12345) and generates a short, unique URL (e.g., https://short.url/aB7x9z2). When clicked, the short link redirects the client browser to the original destination.

Step 1: Clarify Requirements

• Functional:
  1. System accepts a long URL and generates a short unique URL code.
  2. Visiting the short URL redirects the client to the original long URL.
  3. Custom short URLs are out of scope.
  4. Links live indefinitely (no default expiration).
  5. Analytics tracking is a secondary, nice-to-have requirement.
• Non-Functional:
  1. High availability and ultra-low latency redirection ($< 50 \text{ ms}$).
  2. Short links must be non-predictable to prevent brute-force enumeration attacks.

Step 2: Back-of-the-Envelope Estimation

• Write Throughput: 1 million new links created per day.

• Read Throughput: 1,000 clicks per generated link (Read-Heavy 1000:1 ratio).

• 10-Year Storage Scale: 1 million links/day $\times 3650 \text{ days} = 3.65 \text{ billion links}$.
  • Assuming $1 \text{ KB}$ per mapping row (short code + long URL + metadata):

Step 3: Define API & Data Model

API Specifications

POST /api/v1/urls
Content-Type: application/json

{
  "long_url": "https://example.com/deep/path/resource"
}

Response HTTP 201 Created:
{
  "short_code": "aB7x9z2",
  "short_url": "https://short.url/aB7x9z2"
}

GET /{short_code}

Response HTTP 302 Found:
Location: https://example.com/deep/path/resource

Storage Schema

Access patterns involve a simple 1:1 lookup by short_code. Relational storage (e.g., PostgreSQL) with a $B$-tree index on short_code provides high performance and ACID durability.

CREATE TABLE url_mapping (
    short_code VARCHAR(10) PRIMARY KEY,
    long_url TEXT NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_short_code ON url_mapping (short_code);

Step 4: High-Level Design & Unique Code Generation

To create a $7$-character string representing billions of URLs, use Base62 Encoding ([0-9], [a-z], [A-Z]).

Option A: Hashing (MD5 / SHA-256)

Pass long_url into MD5, derive a hash, and extract the first 7 characters.

• Problem: Hash collisions will occur ($H(\text{URL}_1) \text{ and } H(\text{URL}_2)$ produce identical 7-character prefixes). Checking the DB on every write causes latency spikes as the dataset grows.

Option B: Global Counter + Base62 Encoding (Preferred)

Use an auto-incrementing 64-bit integer counter. Convert the integer value directly into Base62.

# Base62 Encoder Implementation
BASE62 = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"

def encode_base62(num: int) -> str:
    if num == 0:
        return BASE62[0]
    arr = []
    base = len(BASE62)
    while num > 0:
        rem = num % base
        arr.append(BASE62[rem])
        num //= base
    return "".join(reversed(arr))

# Example: Counter ID = 11,157,000,000 -> Base62 = "aB7x9z2"

Security Fix: Permutation Shuffling

Problem: Sequential counters produce predictable links (https://short.url/1, https://short.url/2), exposing the system to scraping attacks.

Solution: Apply a reversible bitwise permutation/shuffling function to the counter integer before encoding it to Base62.

def shuffle_counter(counter: int) -> int:
    # Reversible bit manipulation & secret offset
    # Ensures consecutive counters map to non-consecutive Base62 strings
    SECRET_MASK = 0x5555555555555555
    BIT_ROTATION = 13
    shuffled = ((counter ^ SECRET_MASK) << BIT_ROTATION) | ((counter ^ SECRET_MASK) >> (64 - BIT_ROTATION))
    return shuffled & 0xFFFFFFFFFFFFFFFF

5. URL Shortener — Scaling the Read Path & Edge Cases

Because reads outnumber writes $1,000$ to $1$, scale performance by insulating the core database from read traffic.

1. Redis Caching Strategy

Place an in-memory Redis cluster between application servers and PostgreSQL. Because short-code-to-long-URL mappings are immutable (a code never updates its destination), cached mappings never become stale. Configure a Least Recently Used (LRU) eviction policy to retain popular ("hot") links in memory.

2. HTTP Redirection Semantics: 301 vs. 302 Redirects

Engineering Choice: Use 302 Temporary Redirects if link analytics or emergency link deactivation (e.g., blocking malware or phishing links) is required.

3. Distributed Counter Sharding (Eliminating Counter Bottlenecks)

A single centralized counter database is a Single Point of Failure (SPOF) and a throughput bottleneck.

• Solution: Assign each application server a discrete block of pre-allocated counter ranges (e.g., Server 1 takes IDs $1 \text{ to } 1,000,000$; Server 2 takes IDs $1,000,001 \text{ to } 2,000,000$). When a server exhausts its local block, it requests a new range block from a distributed lock coordinator like Apache ZooKeeper.

6. Rate Limiter — Protecting System Boundaries

A Rate Limiter restricts the number of requests a client can make to an API within a given timeframe. It defends against Denial-of-Service (DoS) attacks, brute-force login attempts, resource starvation, and unexpected billing spikes from downstream third-party APIs.

Step 1: Clarify Requirements

• Functional:
  1. Limit clients to 100 API requests per minute.
  2. Identify clients by Account ID (for authenticated users) or IP Address (for unauthenticated users).
  3. When exceeded, return HTTP Status Code 429 (Too Many Requests) accompanied by a Retry-After header.
• Non-Functional:
  1. Ultra-low latency footprint ($< 1 \text{ ms}$ processing overhead per check).
  2. High accuracy; must prevent burst exploit windows.
  3. System must fail open if the rate-limiting infrastructure experiences a widespread failure.

Step 2: Rate Limiting Algorithms Evaluation

1. Fixed Window Counter

Time is divided into fixed intervals (e.g., 1-minute windows). A counter increments per request and resets at the clock boundary.

• Flaw (Boundary Burst Vulnerability): A client can send 100 requests in the last second of Window 1 and 100 requests in the first second of Window 2, resulting in 200 requests within a 2-second span—double the allowed threshold.

2. Sliding Window Log

Stores a log of precise timestamps for every request in a sorted set (e.g., Redis ZSET). On every incoming request, drop all timestamps older than $t - 60 \text{ seconds}$ and count the remaining log size.

• Flaw (High Memory Overhead): Storing a timestamp ($\sim 8 \text{ bytes}$) for every request uses significant memory under high volume. If a limit allows $10,000 \text{ req/min}$, storing $10,000$ timestamps per user across millions of users consumes tens of gigabytes of RAM.

3. Token Bucket Algorithm (Industry Standard)

Each user bucket has a maximum capacity $C$ (e.g., 100 tokens) and fills continuously at a refill rate $R$ (e.g., $1.67 \text{ tokens/second} = 100 \text{ tokens/minute}$). Every request consumes 1 token.

# Token Bucket Data Model per User
class TokenBucket:
    def __init__(self, capacity: int, fill_rate: float):
        self.capacity = capacity       # e.g., 100
        self.fill_rate = fill_rate     # e.g., 1.67 tokens/sec
        self.tokens = capacity
        self.last_update = time.time()

    def allow_request(self) -> bool:
        now = time.time()
        # Lazily recalculate token count based on elapsed time
        elapsed = now - self.last_update
        self.tokens = min(self.capacity, self.tokens + elapsed * self.fill_rate)
        self.last_update = now

        if self.tokens >= 1.0:
            self.tokens -= 1.0
            return True
        return False

• Advantages: Memory efficient ($\sim 50 \text{ bytes}$ per user: tokens_left integer + last_timestamp float). Supports bursts up to capacity $C$ while maintaining a steady rate limit over time.

7. Rate Limiter — Distributed Synchronization & Failure Modes

When rate limiting across multiple API Gateways, localized in-memory counters fail because a load balancer distributes a client's requests across different nodes. A client could bypass the limit entirely if each gateway node maintains its own separate bucket.

Race Conditions & Lua Scripting

Checking and updating a Redis token bucket involves a Read-Calculate-Write sequence. If two requests from the same user hit Gateway 1 and Gateway 2 simultaneously, both gateways could read the same token count, leading to race conditions and inaccurate limits.

To fix this, execute the entire Token Bucket check atomically inside Redis using a Lua Script. Redis executes Lua scripts as single atomic commands, blocking other scripts or commands from running concurrently on the same key.

-- Redis Lua Script for Atomic Token Bucket Check
-- KEYS[1]: User bucket key (e.g., "rate:user_12345")
-- ARGV[1]: Bucket capacity (100)
-- ARGV[2]: Refill rate per second (1.67)
-- ARGV[3]: Current epoch timestamp (seconds)

local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local fill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])

local data = redis.call("HMGET", key, "tokens", "last_update")
local tokens = tonumber(data[1])
local last_update = tonumber(data[2])

if not tokens then
    tokens = capacity
    last_update = now
else
    local elapsed = now - last_update
    tokens = math.min(capacity, tokens + elapsed * fill_rate)
    last_update = now
end

if tokens >= 1 then
    tokens = tokens - 1
    redis.call("HMSET", key, "tokens", tokens, "last_update", last_update)
    return {1, math.floor(tokens)} -- Allowed, return remaining tokens
else
    return {0, math.floor(tokens)} -- Rejected
end

Architectural Failure Modes: Fail-Open vs. Fail-Closed

8. Enterprise Notification System — Asynchronous Architecture

Modern systems deliver notifications via four primary channels: Mobile Push Notifications (Apple APNS, Google FCM), SMS (Twilio, Infobip), Email (SendGrid, AWS SES), and In-App Notifications (Internal database alerts).

Step 1: Clarify Requirements

• Functional:
  1. Support Push, SMS, Email, and In-App notifications.
  2. Respect end-user notification channel preferences (opt-outs).
  3. Differentiate between high-priority Transactional messages (e.g., OTP login codes, payment receipts) and lower-priority Marketing campaigns (e.g., promotional discounts).
• Non-Functional:
  1. Fault tolerance: Third-party outages must not cause data loss.
  2. Low latency for transactional messages ($< 5 \text{ seconds}$).
  3. Minimize duplicate deliveries.

Step 2: The Synchronous Anti-Pattern

A common initial mistake is designing a synchronous notification service where the core application server calls third-party providers directly.

Flaws:

1. Cascade Failure: If SendGrid hangs or experiences high latency, the entire upstream Application Server exhausts its thread pool waiting for a response.
2. High Latency: The client remains blocked for seconds while external network calls complete.
3. No Fault Recovery: If the application server crashes mid-execution, pending notifications are permanently lost.

Step 3: Asynchronous Decoupled Architecture (Per-Channel Cues)

decouple ingestion from delivery using message queues (e.g., Apache Kafka, AWS SQS). Provide dedicated queues for each channel so that a failure or slowdown in one provider (e.g., SendGrid throttling emails) does not impact or delay another channel (e.g., Twilio processing critical SMS messages).

9. Notification System — Reliability & Fault Tolerance

1. The Transactional Outbox Pattern

A common distributed systems bug occurs when an API server writes a notification to PostgreSQL and then attempts to push a job ticket to AWS SQS. If the server crashes after updating the database but before pushing to SQS, the notification is saved as pending but never sent.

To fix this, write the notification and an Outbox Job Record to the database within a single database transaction. A separate background processor periodically polls the outbox table, pushes the pending jobs to SQS, and marks them as processed upon success.

BEGIN TRANSACTION;

INSERT INTO notifications (id, user_id, channel, body, status) 
VALUES ('notif_999', 'usr_42', 'SMS', 'Your code is 4821', 'PENDING');

INSERT INTO outbox_jobs (job_id, notification_id, payload, status) 
VALUES ('job_001', 'notif_999', '{"channel": "SMS", "id": "notif_999"}', 'READY');

COMMIT;

2. Idempotency & At-Least-Once Delivery

Message queues guarantee At-Least-Once Delivery. If a worker receives a ticket, sends an SMS, and then crashes before acknowledging the ticket back to SQS, SQS will re-queue the ticket and deliver it to another worker. This can result in a duplicate SMS being sent to the user.

To prevent duplicate deliveries, use an Idempotency Key (the unique notification_id) stored in Redis with a Time-To-Live (TTL).

def process_sms_notification(worker_id: str, job_ticket: dict):
    notif_id = job_ticket["notification_id"]
    redis_key = f"idempotency:{notif_id}"

    # Atomic SETNX (Set if Not Exists) with a 120-second TTL
    is_acquired = redis_client.set(redis_key, worker_id, nx=True, ex=120)

    if not is_acquired:
        print(f"Notification {notif_id} is already processed/processing. Skipping.")
        sqs_client.delete_message(job_ticket["receipt_handle"])
        return

    try:
        # Call third-party SMS provider
        twilio_client.send_sms(to=job_ticket["phone"], body=job_ticket["body"])
        
        # Update database status to SENT
        db.execute("UPDATE notifications SET status='SENT' WHERE id=%s", (notif_id,))
        
        # Delete message from SQS queue
        sqs_client.delete_message(job_ticket["receipt_handle"])
    except Exception as e:
        # Release idempotency key on explicit failure to allow retries
        redis_client.delete(redis_key)
        raise e

10. Newsfeed System — Fan-Out Strategies

A social media newsfeed (e.g., LinkedIn, X/Twitter, Instagram) presents a aggregated stream of updates published by accounts a user follows, sorted reverse-chronologically or by relevance.

Step 1: Clarify Requirements

• Functional:
  1. Users can publish posts containing text, images, or video links.
  2. Users can follow other users.
  3. Users can view a feed of posts generated by accounts they follow, sorted reverse-chronologically.
  4. Max feed page target latency: $< 1 \text{ second}$.
• Non-Functional:
  1. Workload is Read-Heavy (50 reads per 1 write).
  2. Availability is prioritized over immediate, strict consistency. A new post taking 3–5 seconds to propagate to all follower feeds is acceptable.

Step 2: Back-of-the-Envelope Estimation

• Users: 10 million Daily Active Users (DAU).
• Read Throughput: Each user opens their feed 5 times per day.

• Write Throughput: Users write posts infrequently (average 3 posts per month per user).

Step 3: Fan-Out Architectural Options

Approach A: Fan-Out on Read (Pull Model)

When a post is created, save it to a central posts table. No precomputed feed work is performed on write. When a user requests their feed, dynamically query the database for all posts authored by users they follow.

-- Dynamic Query executed every time a user opens their feed
SELECT p.post_id, p.author_id, p.content, p.created_at
FROM posts p
JOIN follows f ON p.author_id = f.followee_id
WHERE f.follower_id = :current_user_id
ORDER BY p.created_at DESC
LIMIT 50;

• Pros: Simple writes ($O(1)$ database insertion). Zero wasted storage for inactive users who never open their feeds.
• Cons: Extremely expensive reads ($O(N)$ database joins on every feed refresh). Severe latency under high traffic volumes.

Approach B: Fan-Out on Write (Push Model)

Precompute user feeds upon post creation. Maintain a dedicated, ordered feed list for every user in an in-memory store (e.g., Redis ZSET where the score is the post creation timestamp and the value is the post_id).

# Fan-Out on Write Execution Logic
def publish_post(author_id: str, post_id: str, timestamp: float):
    # Retrieve all followers of the author
    followers = db.get_followers(author_id)
    
    # Push post_id to each follower's Redis feed pipeline
    pipeline = redis_cluster.pipeline()
    for follower_id in followers:
        feed_key = f"feed:{follower_id}"
        # ZADD adds post_id scored by timestamp
        pipeline.zadd(feed_key, {post_id: timestamp})
        # Trim list to retain only the top 500 posts
        pipeline.zremrangebyrank(feed_key, 0, -501)
    pipeline.execute()

• Pros: Blazing-fast reads ($O(1)$ query to fetch a precomputed timeline from Redis memory).
• Cons: Write amplification ($O(F)$ operations, where $F$ is the follower count). If a user with millions of followers posts, the background worker must update millions of Redis lists simultaneously.

11. Newsfeed System — Hybrid Architecture & Ranking Pipeline

The Celebrity Problem (Hotspot Fan-Out)

If a high-profile user with 100 million followers (e.g., Lionel Messi) publishes a post using the Fan-Out on Write model, the system must perform 100 million cache writes. This causes severe message queue backlogs, memory exhaustion, and latency spikes across the infrastructure.

The Hybrid Fan-Out Solution

1. Storage Optimization: Limit the precomputed feed size in Redis to 500 post IDs per active user. Store only post_ids in the feed lists. Store the actual post content (text, media links, metadata) in a separate centralized Post Cache.
2. Active User Pruning: Only compute feeds for Active Users (users who have logged in within the last 30 days). Inactive users have their feeds generated dynamically via a background job when they log back into the system.

Transitioning to a ML-Ranked Feed

Modern social networks rank feed items using Machine Learning scoring engines rather than simple reverse-chronological sorting.

12. Architectural Patterns & Comparative Synthesis

Below is a comparative breakdown of key architectural trade-offs, storage paradigms, and scaling bottlenecks across all four systems.

13. Summary and Key Takeaways

1. Focus on Requirements and Trade-Offs: Interviewers evaluate your technical judgment under ambiguity, clear communication of trade-offs, and willingness to collaborate—not memorized buzzwords.
2. Follow a Structured Framework: Use the 6-Step Framework (Clarify, Estimate, APIs, High-Level Design, Deep Dives, Scaling) to structure your time and keep your analysis grounded.
3. Tailor Architecture to Workload: Analyze Read-to-Write ratios early. A $1000:1$ read-heavy workload (e.g., URL Shortener) requires aggressive caching and read replicas, whereas a write-heavy or bursty workload requires message queues.
4. Choose Algorithms Based on Explicit Constraints: For rate limiters, choose Token Buckets to balance burst capacity, throughput, and minimal memory overhead ($\sim 50 \text{ bytes}$ per user). Use atomic operations (e.g., Redis Lua scripts) to avoid race conditions across distributed nodes.
5. Decouple Asynchronous Workflows: Use message queues to isolate distinct operational domains. In notification systems, allocate independent queues per channel (SMS, Push, Email) to prevent third-party outages from cascading across your service.
6. Ensure Idempotency and Fault Tolerance: Design for at-least-once delivery by leveraging idempotency keys stored in Redis. Combine this with the Transactional Outbox pattern to prevent data loss or duplicate state during node crashes.
7. Address Extreme Edge Cases with Hybrid Models: Pure architectural patterns often break at scale. Solve extreme fan-out problems (e.g., the Celebrity Problem in newsfeeds) by combining models: push (Fan-Out on Write) for standard users and pull (Fan-Out on Read) for celebrities.