Chapter 7: SCALING PATTERNS - System Design Course: Complete Interview Preparation | PlayBook Library

SLIDE 46: Scaling Overview

Step 7: Scaling to Instagram’s Scale

Scaling Dimensions:

Every component scales differently:

CODE

#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .error-icon{fill:#552222;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .error-text{fill:#552222;stroke:#552222;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-thickness-normal{stroke-width:1px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .marker{fill:#666;stroke:#666;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .marker.cross{stroke:#666;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 p{margin:0;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .cluster-label text{fill:#333;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .cluster-label span{color:#333;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .cluster-label span p{background-color:transparent;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .label text,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 span{fill:#000000;color:#000000;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node rect,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node circle,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node ellipse,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node polygon,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .rough-node .label text,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node .label text,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .image-shape .label,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .icon-shape .label{text-anchor:middle;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .rough-node .label,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node .label,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .image-shape .label,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .icon-shape .label{text-align:center;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node.clickable{cursor:pointer;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .arrowheadPath{fill:#333333;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .flowchart-link{stroke:#666;fill:none;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edgeLabel{background-color:white;text-align:center;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edgeLabel p{background-color:white;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .cluster text{fill:#333;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .cluster span{color:#333;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 rect.text{fill:none;stroke-width:0;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .icon-shape,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .image-shape{background-color:white;text-align:center;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .icon-shape p,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .image-shape p{background-color:white;padding:2px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .icon-shape .label rect,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 .node .neo-node{stroke:#999;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node rect,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].cluster rect,#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node polygon{stroke:url(#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node path{stroke:url(#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555-gradient);stroke-width:1px;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node circle{stroke:url(#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3d0778cc-e176-4d86-a86b-03e691cb9555 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}Queue TierMedia TierCache TierDatabase TierApplication TierQueue 1Image ProcessingQueue 2Feed GenerationObject StorageInfinite ScaleCDNGlobal DistributionRedis Cluster 1Redis Cluster 2Redis Cluster NShard 1Primary + ReplicasShard 2Primary + ReplicasShard NPrimary + ReplicasAPI Server 1StatelessAPI Server 2StatelessAPI Server NStatelessLoad Balancer

SLIDE 46B: Scaling Overview - Strategies

Scaling Strategies by Tier:

Tier	Strategy	Bottleneck	Solution
Application	Horizontal (add servers)	CPU, Memory	Auto-scale based on load
Database	Sharding + Replication	Storage, QPS	Shard by user_id, read replicas
Cache	Clustering	Memory	Redis cluster, partition by key
Media	Managed service	N/A	S3/Blob auto-scales
Queue	Partitioning	Throughput	Multiple queue partitions

What We’ll Cover:

Application tier scaling (stateless servers, load balancing)
Database tier scaling (sharding, read replicas)
Caching tier scaling (Redis clusters)
Async processing (queue-based workers)
Rate limiting (protecting the system)

SLIDE 47: Application Tier Scaling - Stateless Architecture

Scaling Application Servers (Horizontal Scaling)

Stateless Server Architecture:

Key Principle: Servers store no session state locally

Structure & Concept Flow

User's first request → Server 1 (handles request, no state stored)

User's second request → Server 2 (can handle, doesn't need Server 1's state)

User's third request → Server 1 (works fine, no memory of first request needed)

Why Stateless:

Any server can handle any request
Servers are interchangeable
Can add/remove servers dynamically
Server crashes don’t lose user sessions

SLIDE 47B: Application Tier Scaling - Session Storage

Where Session Data Lives:

Structure & Concept Flow

Session Storage: Redis (centralized)

User logs in → JWT token issued

Token contains: user_id, issued_at, expires_at

Client includes token in every request

Server receives request:

1. Validate JWT signature (stateless, no DB lookup needed)

2. Extract user_id from token

3. Process request

4. Return response

No server-side session storage needed.

SLIDE 47C: Application Tier Scaling - Load Balancing

Load Balancing:

CODE

#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .error-icon{fill:#552222;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .error-text{fill:#552222;stroke:#552222;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-thickness-normal{stroke-width:1px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-thickness-thick{stroke-width:3.5px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-pattern-solid{stroke-dasharray:0;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .marker{fill:#666;stroke:#666;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .marker.cross{stroke:#666;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d p{margin:0;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .cluster-label text{fill:#333;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .cluster-label span{color:#333;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .cluster-label span p{background-color:transparent;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .label text,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d span{fill:#000000;color:#000000;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node rect,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node circle,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node ellipse,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node polygon,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .rough-node .label text,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node .label text,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .image-shape .label,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .icon-shape .label{text-anchor:middle;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .rough-node .label,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node .label,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .image-shape .label,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .icon-shape .label{text-align:center;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node.clickable{cursor:pointer;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .arrowheadPath{fill:#333333;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .flowchart-link{stroke:#666;fill:none;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edgeLabel{background-color:white;text-align:center;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edgeLabel p{background-color:white;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .cluster text{fill:#333;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .cluster span{color:#333;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d rect.text{fill:none;stroke-width:0;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .icon-shape,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .image-shape{background-color:white;text-align:center;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .icon-shape p,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .image-shape p{background-color:white;padding:2px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .icon-shape .label rect,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d .node .neo-node{stroke:#999;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node rect,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].cluster rect,#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node polygon{stroke:url(#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node path{stroke:url(#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d-gradient);stroke-width:1px;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node circle{stroke:url(#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].icon-shape .icon{fill:url(#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-54995ec0-77e7-4f9e-aa03-97f1a944e98d :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}Millions of UsersDNSinstagram.comLoad BalancerLayer 7 HTTPAPI Server 110.0.1.10API Server 210.0.1.11API Server 310.0.1.12API Server N10.0.1.NHealth CheckEvery 10 secRemove from poolRound-robinRound-robinRound-robinRound-robinUnhealthy

SLIDE 47D: Application Tier Scaling - Load Balancing Algorithms

Load Balancing Algorithms:

Algorithm	How It Works	When to Use
Round Robin	Distribute requests sequentially	Equal server capacity, simple
Least Connections	Send to server with fewest active connections	Varying request duration
Weighted Round Robin	Some servers get more traffic (by weight)	Mixed server capacity
IP Hash	Same client IP always goes to same server	Sticky sessions (if needed)

Instagram’s Choice: Least Connections (requests vary in duration - feed vs like)

SLIDE 47E: Application Tier Scaling - Health Checks

Health Checks:

CODE

Load balancer pings each server every 10 seconds:

GET /health
Response: 200 OK
{
  "status": "healthy",
  "uptime": 864000,
  "connections": 245,
  "memory_used_percent": 68
}

If server doesn't respond or returns 503:
- Mark as unhealthy
- Remove from load balancer pool
- Stop sending traffic
- Alert ops team

When server recovers:
- Health check succeeds
- Add back to pool
- Resume receiving traffic

SLIDE 47F: Application Tier Scaling - Auto-Scaling

Auto-Scaling:

CODE

Scaling Triggers:

Scale Up (add servers) when:
- Average CPU > 70% for 5 minutes
- Average memory > 80% for 5 minutes
- Request queue length > 100

Scale Down (remove servers) when:
- Average CPU < 30% for 15 minutes
- Average memory < 50% for 15 minutes

Configuration:
- Minimum instances: 100 (always running)
- Maximum instances: 10,000
- Scale up: Add 20% more servers
- Scale down: Remove 10% of servers (gradual)
- Cooldown: 5 minutes between scaling actions

SLIDE 47G: Application Tier Scaling - Server Specs and Capacity

Server Specifications:

CODE

Per Application Server:
- CPU: 8 cores
- Memory: 32 GB
- Network: 10 Gbps
- Capacity: ~1,000 concurrent requests

For 300,000 peak QPS:
- Assuming 100ms avg request duration
- Concurrent requests: 300,000 × 0.1 = 30,000
- Servers needed: 30,000 / 1,000 = 30 servers minimum
- With redundancy (2x): 60 servers
- With headroom: 100-200 servers deployed

SLIDE 47H: Application Tier Scaling - Deployment

Deployment Strategy (Zero Downtime):

CODE

Rolling Deployment:

1. Deploy to 10% of servers (10 out of 100)
2. Monitor error rates, latency
3. If metrics good: Deploy to next 10%
4. Repeat until all servers updated
5. If issues: Roll back affected servers

Total deployment time: 30-60 minutes
No downtime (always 90% servers available)

SLIDE 48: Database Scaling - Read Replicas (Diagram)

Database Read Scaling: Replication

Problem:

CODE

Feed reads: 100,000 QPS (from capacity estimation)
Single database primary: ~10,000 QPS max
Need 10x more read capacity

Solution: Read Replicas

CODE

#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .error-icon{fill:#552222;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .error-text{fill:#552222;stroke:#552222;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-thickness-normal{stroke-width:1px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .marker{fill:#666;stroke:#666;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .marker.cross{stroke:#666;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 p{margin:0;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .cluster-label text{fill:#333;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .cluster-label span{color:#333;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .cluster-label span p{background-color:transparent;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .label text,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 span{fill:#000000;color:#000000;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node rect,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node circle,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node ellipse,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node polygon,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .rough-node .label text,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node .label text,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .image-shape .label,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .icon-shape .label{text-anchor:middle;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .rough-node .label,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node .label,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .image-shape .label,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .icon-shape .label{text-align:center;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node.clickable{cursor:pointer;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .arrowheadPath{fill:#333333;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .flowchart-link{stroke:#666;fill:none;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edgeLabel{background-color:white;text-align:center;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edgeLabel p{background-color:white;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .cluster text{fill:#333;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .cluster span{color:#333;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 rect.text{fill:none;stroke-width:0;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .icon-shape,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .image-shape{background-color:white;text-align:center;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .icon-shape p,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .image-shape p{background-color:white;padding:2px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .icon-shape .label rect,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 .node .neo-node{stroke:#999;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node rect,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].cluster rect,#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node polygon{stroke:url(#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node path{stroke:url(#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619-gradient);stroke-width:1px;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node circle{stroke:url(#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-9afe748e-e722-4eb6-89eb-7a7afb51c619 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}Write RequestPOST /postsRead Request 1GET /feedRead Request 2GET /posts/123Read Request 3GET /feedPrimary DatabaseRead + Writeus-east-1aRead Replica 1Read Onlyus-east-1bRead Replica 2Read Onlyus-east-1cRead Replica 3Read Onlyus-east-1dAsync Replication100-500ms lagAsync ReplicationAsync Replication

SLIDE 48B: Database Scaling - How Replication Works

How Replication Works:

Write Path:

CODE

1. Application sends INSERT/UPDATE/DELETE to Primary
2. Primary executes query
3. Primary writes to transaction log (WAL - Write-Ahead Log)
4. Primary sends log to Replicas (asynchronously)
5. Replicas apply log entries to their local copy
6. Replicas catch up (100-500ms behind Primary)

Read Path:

Structure & Concept Flow

Application routing logic:

If write query (INSERT, UPDATE, DELETE):

→ Send to Primary

If read query (SELECT):

→ Send to Replica (round-robin)

SLIDE 48C: Database Scaling - Replication Lag

Replication Lag:

Structure & Concept Flow

Typical lag: 100-500 milliseconds

Scenario demonstrating lag:

1. User creates post → Write to Primary (t=0)

2. Replication happens (t=200ms)

3. User immediately requests their posts → Read from Replica (t=100ms)

4. Replica hasn't received the write yet (lag)

5. User doesn't see their new post

Solution: Read-after-write consistency

After write, read from Primary for that user's data (short time window)

SLIDE 48D: Database Scaling - Read-After-Write Pattern

Read-After-Write Consistency Pattern:

CODE

def create_post(user_id, image, caption):
    # Write to primary
    post_id = primary_db.insert(posts, {...})

    # Mark user as "recently wrote"
    cache.set(f"recent_write:{user_id}", true, ttl=5_seconds)

    return post_id

def get_user_posts(user_id):
    # Check if user recently wrote
    if cache.get(f"recent_write:{user_id}"):
        # Read from primary (strong consistency)
        return primary_db.select(posts, where="user_id = ?", user_id)
    else:
        # Read from replica (eventual consistency OK)
        return replica_db.select(posts, where="user_id = ?", user_id)

SLIDE 48E: Database Scaling - Replica Configuration

Replica Configuration:

CODE

For each shard:
- 1 Primary (handles writes + some reads)
- 3 Read Replicas (handle most reads)

Read distribution:
- Primary: 10% of reads (read-after-write cases)
- Replica 1: 30% of reads
- Replica 2: 30% of reads
- Replica 3: 30% of reads

Example: Shard 0
- Primary (Shard 0): 10,000 write QPS + 1,000 read QPS
- Replica 1 (Shard 0): 3,000 read QPS
- Replica 2 (Shard 0): 3,000 read QPS
- Replica 3 (Shard 0): 3,000 read QPS
Total capacity: 10,000 read QPS per shard

SLIDE 48F: Database Scaling - Failover

Failover (Disaster Recovery):

CODE

If Primary fails:
1. Monitoring detects Primary is down (10-30 seconds)
2. Promote Replica 1 to Primary
3. Update application config (redirect writes to new Primary)
4. Spin up new Replica to replace promoted one
5. Total downtime: 1-2 minutes

Total Database Infrastructure:

CODE

1,000 shards × 4 instances (1 primary + 3 replicas) = 4,000 database instances

Cost per instance: $500/month (r5.2xlarge equivalent)
Total database cost: 4,000 × $500 = $2,000,000/month

This is significant but necessary at Instagram's scale.

SLIDE 49: Database Sharding Revisited - Why Shard

Database Write Scaling: Sharding

Recap: Why Sharding:

CODE

Storage limits:
- Single PostgreSQL instance: ~10 TB practical limit
- Instagram's data: 36 TB metadata/year + historical data
- Need multiple databases

Write throughput limits:
- Single instance: ~10,000 writes/sec
- Instagram write load: 1,200 writes/sec (manageable)
- But with growth: Need headroom

Read throughput limits (solved by replicas, but...):
- Even with replicas, single primary becomes bottleneck for writes

SLIDE 49B: Database Sharding Revisited - Scheme Details

Sharding Scheme (Detailed):

Structure & Concept Flow

Total: 1,000 shards (shard_0 to shard_999)

Shard determination:

shard_id = user_id % 1000

Examples:

- user_id 12345 → shard 345

- user_id 67890 → shard 890

- user_id 999999 → shard 999

Each shard contains:

- ~2 million users (2B users / 1000 shards)

- ~200 million posts (100B posts / 1000 shards, assuming equal distribution)

- Follows, Likes, Comments for those users

SLIDE 49C: Database Sharding Revisited - Data Co-Location

Data Co-Location:

Users Table:

Structure & Concept Flow

Sharded by user_id (directly)

User 12345 → Shard 345

Posts Table:

Structure & Concept Flow

Sharded by user_id (post owner's user_id, not post_id)

Post by User 12345 → Shard 345

Why: "Get all posts by user" query stays on single shard

Follows Table:

Structure & Concept Flow

Sharded by follower_id

"User 12345 follows User 67890" → Shard 345

Why: "Get users I follow" query stays on single shard

Likes and Comments:

Structure & Concept Flow

Sharded by post owner's user_id

Like on post by User 12345 → Shard 345

Why: "Get likes for my posts" stays on single shard

SLIDE 49D: Database Sharding Revisited - Query Patterns

Query Patterns:

Single-Shard Queries (Fast):

CODE

-- Get user's posts (stays on one shard)
SELECT * FROM posts WHERE user_id = 12345
Route to: Shard 345 only

-- Get users I follow (stays on one shard)
SELECT * FROM follows WHERE follower_id = 12345
Route to: Shard 345 only

-- Get likes on my post (stays on one shard)
SELECT * FROM likes WHERE post_id IN (
  SELECT post_id FROM posts WHERE user_id = 12345
)
Route to: Shard 345 only

Cross-Shard Queries (Slow, Avoid):

CODE

-- Who follows me? (followers could be on ANY shard)
SELECT * FROM follows WHERE followee_id = 12345
Problem: Must query ALL 1,000 shards

Solution: Denormalize or cache this data

SLIDE 49E: Database Sharding Revisited - Shard Router

Shard Router (Application Layer):

Structure & Concept Flow

class ShardRouter:

def __init__(self):

self.num_shards = 1000

self.shard_connections = {} # shard_id -> database connection

def get_shard(self, user_id):

shard_id = user_id % self.num_shards

return self.shard_connections[shard_id]

def execute_query(self, user_id, query):

shard = self.get_shard(user_id)

return shard.execute(query)

# Usage

router = ShardRouter()

posts = router.execute_query(

user_id=12345,

query="SELECT * FROM posts WHERE user_id = 12345"

)

SLIDE 49F: Database Sharding Revisited - Re-Sharding

Adding New Shards (Re-Sharding):

Structure & Concept Flow

When to re-shard:

- Individual shard exceeds 10 TB

- Write throughput on shard exceeds capacity

- Typically every 2-3 years

How to re-shard:

1. Double number of shards: 1,000 → 2,000

2. For each user:

- Old: shard_id = user_id % 1,000

- New: shard_id = user_id % 2,000

- If different shard, migrate data

3. Some users stay on same shard (user_id 0, 2000, 4000, ... stay on shard 0)

4. Some users move to new shards

Migration strategy:

- Background job migrates 1% of users per day (100-day migration)

- Application reads from both old and new shards during migration

- Once migration complete, decommission old shards

Advanced: Consistent hashing reduces data movement during re-sharding

SLIDE 50: Caching Strategy - Multi-Layer Overview

Multi-Layer Caching

Cache Hierarchy:

CODE

#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .error-icon{fill:#552222;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .error-text{fill:#552222;stroke:#552222;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-thickness-normal{stroke-width:1px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-thickness-thick{stroke-width:3.5px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-pattern-solid{stroke-dasharray:0;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .marker{fill:#666;stroke:#666;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .marker.cross{stroke:#666;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f p{margin:0;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .cluster-label text{fill:#333;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .cluster-label span{color:#333;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .cluster-label span p{background-color:transparent;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .label text,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f span{fill:#000000;color:#000000;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node rect,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node circle,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node ellipse,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node polygon,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .rough-node .label text,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node .label text,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .image-shape .label,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .icon-shape .label{text-anchor:middle;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .rough-node .label,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node .label,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .image-shape .label,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .icon-shape .label{text-align:center;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node.clickable{cursor:pointer;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .arrowheadPath{fill:#333333;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .flowchart-link{stroke:#666;fill:none;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edgeLabel{background-color:white;text-align:center;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edgeLabel p{background-color:white;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .cluster text{fill:#333;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .cluster span{color:#333;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f rect.text{fill:none;stroke-width:0;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .icon-shape,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .image-shape{background-color:white;text-align:center;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .icon-shape p,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .image-shape p{background-color:white;padding:2px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .icon-shape .label rect,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f .node .neo-node{stroke:#999;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node rect,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].cluster rect,#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node polygon{stroke:url(#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node path{stroke:url(#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f-gradient);stroke-width:1px;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node circle{stroke:url(#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].icon-shape .icon{fill:url(#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-3fce589c-92f0-43dc-8713-bdbbd750637f :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}Client RequestL1: Client CacheMobile App Local StorageL2: CDN CacheEdge Locations WorldwideL3: Application CacheRedis - Feed, User DataL4: Database Query CachePostgreSQL InternalDatabaseReturn0ms networkReturn20-50msReturn10-30msReturn50-100msReturn200-500msMissMissMissMissHit 30%Hit 50%Hit 15%Hit 4%Miss 1%

SLIDE 50B: Caching Strategy - L1 Client Cache

L1: Client-Side Cache (Mobile App)

CODE

What's cached:
- User's own feed (last 100 posts)
- User's own posts
- Recently viewed posts
- User profiles visited

Storage location:
- iOS: Core Data, NSCache
- Android: Room Database, SharedPreferences

TTL: Until app restart or explicit refresh

Benefits:
- Instant load (0ms network latency)
- Offline viewing
- Reduced data usage

Invalidation:
- User pulls to refresh
- App restarted
- Explicit cache clear

SLIDE 50C: Caching Strategy - L2 CDN and L3 Redis

L2: CDN Cache (Media Files)

CODE

Covered earlier in Media Storage section.

What's cached: Images, videos
TTL: 1 year (immutable URLs)
Hit rate: 80-85%

L3: Application Cache (Redis)

What’s Cached:

Data Type	Key Pattern	TTL	Size	Purpose
User Feed	`feed:{user_id}`	5 min	1 KB	Pre-computed feed
User Profile	`user:{user_id}`	1 hour	500 bytes	Avoid user table queries
Post Metadata	`post:{post_id}`	1 hour	1 KB	Avoid post table queries
Follower Count	`count:followers:{user_id}`	10 min	8 bytes	Aggregated count
Like Count	`count:likes:{post_id}`	1 min	8 bytes	Real-time count

SLIDE 50D: Caching Strategy - Redis Cluster

Redis Cluster Configuration:

Structure & Concept Flow

Setup:

- 100 Redis instances

- Each instance: 32 GB RAM

- Total cache capacity: 3.2 TB

- Partitioned by key (consistent hashing)

Replication:

- 3 replicas per master

- 100 masters + 300 replicas = 400 Redis instances total

Failover:

- Master fails → Automatic promotion of replica

- Redis Sentinel monitors health

- Sub-second failover

SLIDE 50E: Caching Strategy - Eviction and Warming

Cache Eviction Policies:

CODE

Policy: LRU (Least Recently Used)

When memory full:
1. Identify least recently accessed keys
2. Evict those keys
3. Make room for new data

Example:
- User hasn't loaded app in 6 months
- Their feed cache hasn't been accessed
- Evicted to make room for active users

Cache Warming (Pre-Loading):

CODE

For VIP users (celebrities):
- Pre-compute feed every 5 minutes (before cache expiration)
- Ensures their feed is always cached
- They never experience cache miss

Background job:
Every 5 minutes:
  FOR each VIP user:
    IF cache expiring soon:
      Recompute feed
      Update cache

SLIDE 50F: Caching Strategy - L4 Database Cache

L4: Database Query Cache (PostgreSQL)

CODE

PostgreSQL internal caching:
- Query plan cache (parsed queries)
- Shared buffer cache (frequently accessed pages)
- OS page cache (file system level)

Configured in postgresql.conf:
shared_buffers = 32GB  (frequently accessed data)
effective_cache_size = 96GB  (total RAM for caching)

Benefit: Repeated queries are faster (no disk I/O)

SLIDE 50G: Caching Strategy - Invalidation Strategies

Cache Invalidation Strategy:

Write-Through Cache:

CODE

def update_user_profile(user_id, new_bio):
    # 1. Update database
    db.update("UPDATE users SET bio = ? WHERE user_id = ?", new_bio, user_id)

    # 2. Update cache
    cache.delete(f"user:{user_id}")  # Invalidate old cache

    # 3. Optionally: Pre-warm cache
    user = db.select("SELECT * FROM users WHERE user_id = ?", user_id)
    cache.set(f"user:{user_id}", user, ttl=3600)

Lazy Invalidation:

CODE

def delete_post(post_id):
    # Just delete from database
    db.delete("DELETE FROM posts WHERE post_id = ?", post_id)

    # Don't invalidate caches immediately (expensive)
    # Let them expire naturally (TTL)
    # On cache hit, check if post exists, if not, cache miss

SLIDE 51: Async Processing - Why and Use Cases

Asynchronous Processing

Why Async:

Structure & Concept Flow

Synchronous (Bad for expensive operations):

User uploads photo → Server processes (10 seconds) → User waits → Response

Asynchronous (Good):

User uploads photo → Server queues job (100ms) → User gets 202 Accepted → Processing happens in background

Use Cases for Async Processing:

Operation	Why Async	Processing Time
Image Processing	Thumbnail generation CPU-intensive	5-10 seconds
Feed Fan-Out	Writing to millions of follower caches	10-60 seconds
Email Notifications	Email sending slow, third-party API	1-5 seconds
Analytics Events	Not time-critical, batch processing	N/A
Push Notifications	Third-party service (APNs, FCM)	1-3 seconds

SLIDE 51B: Async Processing - Architecture

Queue Architecture:

CODE

#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .error-icon{fill:#552222;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .error-text{fill:#552222;stroke:#552222;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-thickness-normal{stroke-width:1px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .marker{fill:#666;stroke:#666;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .marker.cross{stroke:#666;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 p{margin:0;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .cluster-label text{fill:#333;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .cluster-label span{color:#333;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .cluster-label span p{background-color:transparent;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .label text,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 span{fill:#000000;color:#000000;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node rect,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node circle,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node ellipse,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node polygon,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .rough-node .label text,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node .label text,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .image-shape .label,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .icon-shape .label{text-anchor:middle;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .rough-node .label,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node .label,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .image-shape .label,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .icon-shape .label{text-align:center;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node.clickable{cursor:pointer;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .arrowheadPath{fill:#333333;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .flowchart-link{stroke:#666;fill:none;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edgeLabel{background-color:white;text-align:center;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edgeLabel p{background-color:white;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .cluster text{fill:#333;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .cluster span{color:#333;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 rect.text{fill:none;stroke-width:0;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .icon-shape,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .image-shape{background-color:white;text-align:center;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .icon-shape p,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .image-shape p{background-color:white;padding:2px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .icon-shape .label rect,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 .node .neo-node{stroke:#999;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node rect,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].cluster rect,#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node polygon{stroke:url(#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node path{stroke:url(#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190-gradient);stroke-width:1px;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node circle{stroke:url(#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b63931ff-625d-46ae-88d2-86dcf0a82190 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}API ServerImage QueueAWS SQSFeed QueueAWS SQSNotification QueueAWS SQSImage Worker 1Image Worker 2Feed Worker 1Notification WorkerProcess & StoreUpdate CachesSend PushEnqueueEnqueueEnqueue

SLIDE 51C: Async Processing - Technology Options

Message Queue Technology:

Option	Use Case	Throughput
AWS SQS	General purpose, managed	10K+ msgs/sec per queue
Azure Service Bus	Azure-native, similar to SQS	10K+ msgs/sec
RabbitMQ	Self-hosted, flexible routing	20K+ msgs/sec
Apache Kafka	High throughput, streaming	1M+ msgs/sec

Instagram’s Choice: Combination

SQS for most jobs (managed, reliable)
Kafka for high-throughput analytics events

SLIDE 51D: Async Processing - Message Format

Queue Message Format:

CODE

Image Processing Job:
{
  "job_id": "abc123",
  "job_type": "process_image",
  "post_id": 789456,
  "user_id": 12345,
  "original_image_url": "s3://temp/789456_upload.jpg",
  "operations": ["compress", "generate_thumbnails"],
  "callback_url": "/api/v1/posts/789456/complete",
  "created_at": "2025-01-15T10:30:00Z",
  "priority": "normal"
}

SLIDE 51E: Async Processing - Worker Logic

Worker Processing:

CODE

class ImageWorker:
    def run(self):
        while True:
            # 1. Poll queue for messages
            messages = queue.receive(max_messages=10, wait_time=20)

            for message in messages:
                try:
                    # 2. Parse job
                    job = json.loads(message.body)

                    # 3. Process image
                    self.process_image(job)

                    # 4. Delete message (mark as complete)
                    queue.delete(message)

                except Exception as e:
                    # 5. On error: Leave message in queue
                    # Queue will retry after visibility timeout
                    log_error(e)

SLIDE 51F: Async Processing - Retry Logic

Retry Logic:

CODE

Queue Visibility Timeout: 30 seconds

Normal flow:
1. Worker receives message
2. Message becomes invisible to other workers (30 sec timeout)
3. Worker processes (10 sec)
4. Worker deletes message (marked complete)

Failure flow:
1. Worker receives message
2. Message invisible (30 sec timeout starts)
3. Worker crashes (doesn't delete message)
4. After 30 seconds: Message becomes visible again
5. Another worker picks it up (automatic retry)

Max retries: 3 attempts
After 3 failures: Move to Dead Letter Queue (manual inspection)

SLIDE 51G: Async Processing - Scaling and Priority

Worker Auto-Scaling:

CODE

Scale workers based on queue length:

If queue length > 1000 messages:
  - Scale up workers (add 20%)

If queue length < 100 messages for 10 minutes:
  - Scale down workers (remove 10%)

Configuration:
- Min workers: 100 (always running)
- Max workers: 10,000
- Target: Process all messages within 1 minute

Priority Queues:

CODE

For VIP users or urgent jobs:

High Priority Queue:
- Celebrity posts (need fast processing)
- Verified users
- Process first

Normal Priority Queue:
- Regular users
- Process after high priority queue empty

Implementation:
Workers poll high priority queue first, then normal queue

SLIDE 52: Rate Limiting - Why and Tiers

Rate Limiting and Throttling

Why Rate Limiting:

CODE

Protect system from:
1. Abusive users (bots scraping data)
2. Buggy clients (infinite loops making requests)
3. DDoS attacks
4. Unfair resource usage (one user overwhelming system)

Rate Limit Tiers:

Per-User Limits:

Endpoint	Rate Limit	Window	Reasoning
POST /posts	10/hour	Rolling 1 hour	Prevent spam posting
POST /comments	60/hour	Rolling 1 hour	Allow engagement, prevent spam
POST /likes	1000/hour	Rolling 1 hour	High limit for normal usage
GET /feed	120/hour	Rolling 1 hour	2 per minute = reasonable
GET /users/{id}	300/hour	Rolling 1 hour	Profile browsing
POST /follows	200/hour	Rolling 1 hour	Prevent aggressive following

SLIDE 52B: Rate Limiting - Per-IP Limits

Per-IP Limits (Broader Protection):

Scope	Rate Limit	Window
All Endpoints Combined	10,000 requests/hour	Rolling 1 hour
Authentication Endpoints	20 attempts/minute	Rolling 1 minute
Signup	10 accounts/day	Rolling 24 hours

SLIDE 52C: Rate Limiting - Token Bucket Algorithm

Implementation: Token Bucket Algorithm

Token Bucket Concept:

CODE

Bucket: Holds tokens (capacity = rate limit)
Refill rate: Add 1 token per minute (for 60/hour limit)

Request comes in:
- If bucket has token: Consume token, allow request
- If bucket empty: Reject request (429 Too Many Requests)

Example (10 requests/hour limit):
Bucket capacity: 10 tokens
Refill rate: 10 tokens / 60 minutes = 1 token every 6 minutes

User makes 10 requests rapidly: All succeed (10 tokens consumed)
User makes 11th request: Rejected (no tokens left)
After 6 minutes: 1 token added back
User can make 1 more request

SLIDE 52D: Rate Limiting - Redis Implementation

Redis Implementation:

CODE

def check_rate_limit(user_id, endpoint, limit, window):
    """
    Token bucket rate limiter using Redis

    Args:
        user_id: User identifier
        endpoint: API endpoint
        limit: Max requests allowed
        window: Time window in seconds
    """
    key = f"rate_limit:{user_id}:{endpoint}"

    # Get current token count
    current = redis.get(key)

    if current is None:
        # First request: Initialize bucket
        redis.setex(key, window, limit - 1)
        return True  # Allow request

    if int(current) > 0:
        # Tokens available: Consume one
        redis.decr(key)
        return True  # Allow request
    else:
        # No tokens: Reject
        ttl = redis.ttl(key)
        return False, f"Rate limit exceeded. Retry after{ttl} seconds"

# Usage in API
if not check_rate_limit(user_id, "/posts", limit=10, window=3600):
    return Response(429, {
        "error": "rate_limit_exceeded",
        "message": "You can only create 10 posts per hour",
        "retry_after": 1800  # seconds
    })

SLIDE 52E: Rate Limiting - Sliding Window

Sliding Window Implementation (More Accurate):

CODE

def sliding_window_rate_limit(user_id, endpoint, limit, window):
    """
    More accurate than token bucket
    Counts actual requests in rolling window
    """
    key = f"rate_limit:{user_id}:{endpoint}"
    now = time.time()
    window_start = now - window

    # Use Redis sorted set (timestamp as score)
    pipe = redis.pipeline()

    # Remove old requests outside window
    pipe.zremrangebyscore(key, 0, window_start)

    # Count requests in current window
    pipe.zcard(key)

    # Add current request
    pipe.zadd(key, {str(now): now})

    # Set expiration
    pipe.expire(key, window)

    results = pipe.execute()
    request_count = results[1]

    if request_count < limit:
        return True  # Allow
    else:
        return False  # Reject

SLIDE 52F: Rate Limiting - Response Headers and Bypasses

Response Headers:

CODE

HTTP/1.1 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1736956800

When rate limit exceeded:
HTTP/1.1 429 Too Many Requests
Retry-After: 1800
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1736956800

{
  "error": "rate_limit_exceeded",
  "message": "API rate limit exceeded for POST /posts",
  "retry_after_seconds": 1800
}

Bypasses for Trusted Clients:

CODE

VIP users (verified accounts, Instagram staff):
- Higher limits (100x normal limits)
- Separate rate limit pool

Internal services:
- No rate limits
- Authenticated with service tokens

Mobile apps:
- More lenient than web (better UX)
- 2x limits compared to web