Chapter 6: MEDIA STORAGE & CDN - System Design Course: Complete Interview Preparation | PlayBook Library

SLIDE 41: Media Storage Challenge - The Problem

Step 6: Media Storage and Delivery

The Scale Problem:

Storage Requirements:

Metric	Value	Calculation
Photos Uploaded Daily	100 million	Given in requirements
Average Photo Size	2 MB	After compression
Daily Storage Growth	200 TB	100M × 2MB
Annual Storage Growth	73 PB	200 TB × 365 days
5-Year Storage	365 PB	73 PB × 5 years

SLIDE 41B: Media Storage Challenge - Delivery Requirements

Delivery Requirements:

Metric	Value	Calculation
Daily Photo Views	10 billion	500M users × 20 views/day
Egress Bandwidth	231 GB/second	10B × 2MB / 86,400 sec
Peak Bandwidth	693 GB/second	3x average (peak hours)

SLIDE 41C: Media Storage Challenge - Failed Approaches

Why Traditional Approaches Fail:

Approach 1: Store in Database (Bad)

CODE

CREATE TABLE photos (
    photo_id BIGINT,
    image_blob BLOB  -- 2MB per row
);

Problems:
- Database not designed for large binary data
- Backups become massive and slow
- Queries slow (must skip large blob columns)
- Replication costly (replicating 73 PB/year)
- Cannot leverage CDN (data locked in database)

Approach 2: Store on Web Servers (Bad)

CODE

/var/www/images/photo_123.jpg

Problems:
- Web servers have limited disk space
- Cannot scale horizontally (which server has which photo?)
- No redundancy (server crash = data loss)
- 231 GB/sec from origin servers = impossible

SLIDE 41D: Media Storage Challenge - Requirements

Requirements for Solution:

Requirement	Why Critical
Massive Scale	Must handle petabytes of data
High Durability	Cannot lose user photos (99.999999999% durability)
Global Delivery	Users worldwide need low latency
Cost-Effective	Storing 365 PB at $25/TB/month =$ 9M/month (prohibitive)
Fast Upload	Users won’t wait 30 seconds to upload
Multiple Formats	Need original + 3 thumbnail sizes

Solution Architecture:

CODE

Object Storage (S3/Azure Blob) + CDN + Async Processing

Components:
1. Object Storage: Durable, scalable storage (S3, Azure Blob, Google Cloud Storage)
2. CDN: Global edge caching (CloudFront, Akamai, Azure CDN)
3. Processing Pipeline: Async thumbnail generation, compression
4. Tiered Storage: Hot (recent) vs Cold (old) storage for cost optimization

SLIDE 42: Object Storage - Why and Features

Object Storage: S3 / Azure Blob Storage

Why Object Storage:

Feature	Benefit
Designed for Scale	Exabyte-scale storage capacity
High Durability	99.999999999% (11 nines) - lose 1 object per 10,000 years
Built-in Redundancy	Data replicated across multiple data centers automatically
Cost-Effective	$0.023/GB/month (S3 Standard), cheaper tiers available
HTTP-Accessible	Direct URL access, no custom protocol needed
Lifecycle Policies	Automatically move old data to cheaper storage tiers

SLIDE 42B: Object Storage - Hierarchy

Storage Hierarchy:

CODE

Bucket: instagram-media-production-us-east-1

/posts/
  /original/
    /2025/
      /01/
        /15/
          789456_original.jpg (2MB)
          789457_original.jpg (2MB)
  /thumbnails/
    /small/
      /2025/
        /01/
          /15/
            789456_150x150.jpg (20KB)
            789457_150x150.jpg (20KB)
    /medium/
      /2025/
        /01/
          /15/
            789456_640x640.jpg (150KB)
    /large/
      /2025/
        /01/
          /15/
            789456_1080x1080.jpg (500KB)

SLIDE 42C: Object Storage - Naming and URLs

File Naming Convention:

CODE

Format: {post_id}_{size}.jpg

Examples:
- 789456_original.jpg
- 789456_150x150.jpg
- 789456_640x640.jpg
- 789456_1080x1080.jpg

Benefits:
- Predictable URLs (can construct without database lookup)
- Easy to identify which post
- Size included in filename

URL Structure:

CODE

Original:
https://s3.amazonaws.com/instagram-media-prod/posts/original/2025/01/15/789456_original.jpg

After CDN:
https://cdn.instagram.com/posts/789456_1080x1080.jpg

Shortened in database:
/posts/789456_1080x1080.jpg (store relative path, construct full URL in application)

SLIDE 42D: Object Storage - Partitioning

Date-Based Partitioning:

Why Partition by Date:

Easier lifecycle management (move all of 2020’s photos to cold storage)
Better S3 performance (avoids hot-spotting single prefix)
Simpler to reason about growth (X TB per month)
Compliance (delete all data before date Y if required)

SLIDE 42E: Object Storage - Tiers

Storage Tiers (Cost Optimization):

Tier	Use Case	Cost	Retrieval Time
S3 Standard	Recent photos (<30 days)	$0.023/GB/month	Instant
S3 Intelligent-Tiering	Photos 30-90 days old	$0.023-0.0125/GB/month	Instant
S3 Standard-IA	Photos 90 days-1 year old	$0.0125/GB/month	Instant
S3 Glacier Instant	Photos 1-3 years old	$0.004/GB/month	Instant
S3 Glacier Deep Archive	Photos >3 years old	$0.00099/GB/month	12 hours

SLIDE 42F: Object Storage - Lifecycle Policy

Lifecycle Policy Example:

CODE

Automatically transition data:
- Day 0-30: S3 Standard
- Day 30-90: S3 Intelligent-Tiering
- Day 90-365: S3 Standard-IA (Infrequent Access)
- Day 365+: S3 Glacier Instant Retrieval

5-year storage cost calculation:
Year 1 (73 PB): 73,000 TB × $23/TB = $1,679,000/month
Year 2 (73 PB): Mostly in IA tier: 73,000 × $12.50/TB = $912,500/month
Year 3+ (73 PB): Mostly in Glacier: 73,000 × $4/TB = $292,000/month

Total over 5 years: ~$200M (vs $500M+ if all stayed in S3 Standard)

Azure Blob Storage Equivalent:

For organizations using Azure:

Hot tier (S3 Standard equivalent)
Cool tier (S3 IA equivalent)
Archive tier (S3 Glacier equivalent)
Similar pricing and lifecycle policies

SLIDE 43: Image Processing Pipeline (Diagram)

Image Upload and Processing Pipeline

CODE

#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .error-icon{fill:#552222;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .error-text{fill:#552222;stroke:#552222;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-thickness-normal{stroke-width:1px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .marker{fill:#666;stroke:#666;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .marker.cross{stroke:#666;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 p{margin:0;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .cluster-label text{fill:#333;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .cluster-label span{color:#333;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .cluster-label span p{background-color:transparent;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .label text,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 span{fill:#000000;color:#000000;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node rect,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node circle,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node ellipse,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node polygon,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .rough-node .label text,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node .label text,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .image-shape .label,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .icon-shape .label{text-anchor:middle;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .rough-node .label,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node .label,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .image-shape .label,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .icon-shape .label{text-align:center;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node.clickable{cursor:pointer;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .arrowheadPath{fill:#333333;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .flowchart-link{stroke:#666;fill:none;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edgeLabel{background-color:white;text-align:center;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edgeLabel p{background-color:white;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .cluster text{fill:#333;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .cluster span{color:#333;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 rect.text{fill:none;stroke-width:0;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .icon-shape,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .image-shape{background-color:white;text-align:center;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .icon-shape p,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .image-shape p{background-color:white;padding:2px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .icon-shape .label rect,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 .node .neo-node{stroke:#999;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node rect,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].cluster rect,#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node polygon{stroke:url(#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node path{stroke:url(#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70-gradient);stroke-width:1px;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node circle{stroke:url(#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-62868c00-e0e2-4c72-a87c-2e3b18871a70 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}User Uploads 5MB PhotoAPI GatewayMedia ServiceProcessing QueueAWS SQS / Azure Service BusImage Processing WorkerCompress Original5MB to 2MBJPEG Quality 85%Generate ThumbnailsSmall: 150x15020KBMedium: 640x640150KBLarge: 1080x1080500KBObject StorageS3 / Azure BlobPost Metadata DBCDNNotify UserWebSocket/Push202 Acceptedjob_id: abc123

SLIDE 43B: Image Processing - Steps 1-3

Processing Steps in Detail:

Step 1: Upload Initiation (User Side)

CODE

Client sends:
POST /v1/posts
Content-Type: multipart/form-data

Parts:
- image: [5MB JPEG file]
- caption: "Beautiful day!"
- location: "Oxford, UK"

Client behavior:
- Show upload progress bar
- Display "Processing..." message
- Don't block UI

Step 2: API Gateway Receives Upload (50-200ms)

CODE

Actions:
1. Validate authentication (JWT token)
2. Check file size (<10MB limit)
3. Validate file type (JPEG, PNG only)
4. Generate unique post_id and job_id
5. Forward to Media Service

Step 3: Media Service Enqueues Job (10-20ms)

CODE

Media Service:
1. Store original file in temporary storage (S3 temp bucket)
2. Create job in queue:
   {
     job_id: "abc123",
     post_id: 789456,
     user_id: 12345,
     original_url: "s3://temp/789456_upload.jpg",
     operations: ["compress", "generate_thumbnails"]
   }
3. Return 202 Accepted to client

SLIDE 43C: Image Processing - Steps 4-6

Step 4: Worker Picks Up Job (5-10 seconds later)

CODE

Worker (background process):
1. Poll queue for jobs
2. Receive job message
3. Download original image from temp storage
4. Process image (next steps)

Step 5: Image Compression (2-3 seconds)

Structure & Concept Flow

Using ImageMagick or similar:

convert original.jpg \

-quality 85 \

-strip (remove EXIF metadata) \

-sampling-factor 4:2:0 (chroma subsampling) \

compressed.jpg

Result: 5MB → 2MB (60% reduction, minimal quality loss)

Step 6: Thumbnail Generation (3-4 seconds)

CODE

Generate 3 sizes in parallel:

Small (Profile/Grid):
convert compressed.jpg -resize 150x150^ -gravity center -extent 150x150 small.jpg
Output: 20KB

Medium (Feed Mobile):
convert compressed.jpg -resize 640x640^ -gravity center -extent 640x640 medium.jpg
Output: 150KB

Large (Feed Desktop):
convert compressed.jpg -resize 1080x1080^ -gravity center -extent 1080x1080 large.jpg
Output: 500KB

Note: ^ forces exact dimensions, -gravity center crops from center

SLIDE 43D: Image Processing - Steps 7-10

Step 7: Upload to Permanent Storage (2-3 seconds)

CODE

Upload 4 files to S3:
1. s3://instagram-media/posts/original/2025/01/15/789456_original.jpg (2MB)
2. s3://instagram-media/posts/thumbnails/small/.../789456_150x150.jpg (20KB)
3. s3://instagram-media/posts/thumbnails/medium/.../789456_640x640.jpg (150KB)
4. s3://instagram-media/posts/thumbnails/large/.../789456_1080x1080.jpg (500KB)

Parallel uploads, total time: ~3 seconds

Step 8: Create Post Metadata (50-100ms)

CODE

INSERT INTO posts (
  post_id, user_id, image_url,
  thumbnail_small_url, thumbnail_medium_url, thumbnail_large_url,
  caption, location, created_at
) VALUES (
  789456, 12345,
  '/posts/original/.../789456_original.jpg',
  '/posts/thumbnails/small/.../789456_150x150.jpg',
  '/posts/thumbnails/medium/.../789456_640x640.jpg',
  '/posts/thumbnails/large/.../789456_1080x1080.jpg',
  'Beautiful day!', 'Oxford, UK', NOW()
);

SLIDE 43E: Image Processing - Steps 9-10 and Timing

Step 9: CDN Invalidation (100ms)

CODE

Optional: Pre-warm CDN cache
Send request to CDN to cache new images at edge locations

For most cases: Lazy loading (cache on first request) is sufficient

Step 10: Notify User (50ms)

Structure & Concept Flow

WebSocket message to client:

{

"event": "post_published",

"post_id": 789456,

"status": "success",

"urls": {

"original": "https://cdn.instagram.com/posts/789456_original.jpg",

"large": "https://cdn.instagram.com/posts/789456_1080x1080.jpg"

}

Client updates UI: "Processing..." → "Posted!"

Total Processing Time:

Compression: 2-3 seconds
Thumbnail generation: 3-4 seconds
S3 upload: 2-3 seconds
Database insert: 0.1 seconds
Total: 7-10 seconds

User sees “Posted!” after ~10 seconds, but upload API returned in <1 second.

SLIDE 43F: Image Processing - Error Handling

Error Handling:

CODE

If processing fails:
1. Retry job 3 times (queue auto-retry)
2. If still failing, mark as failed
3. Notify user: "Upload failed, please try again"
4. Log error for debugging
5. Clean up temp files

SLIDE 43G: Image Processing - Scaling Workers

Scaling Workers:

CODE

Worker Pool Configuration:
- 100 workers initially
- Each worker processes 6 jobs/minute (10 seconds per job)
- Capacity: 100 × 6 = 600 jobs/minute = 36,000 jobs/hour

Peak upload rate: 3,000 QPS (from capacity estimation)
- 3,000 jobs/sec × 60 = 180,000 jobs/minute
- Need: 180,000 / 6 = 30,000 workers at peak

Solution: Auto-scaling
- Normal hours: 1,000 workers
- Peak hours: 30,000 workers (scale up based on queue length)

SLIDE 44: CDN Architecture - Why CDN

Content Delivery Network (CDN)

Why CDN is Mandatory (Not Optional):

Problem Without CDN:

Structure & Concept Flow

Users request images from origin (S3 in us-east-1):

- User in Tokyo → us-east-1: 200ms latency

- User in Sydney → us-east-1: 250ms latency

- User in London → us-east-1: 100ms latency

- 231 GB/sec egress from origin = impossible and expensive

Solution With CDN:

Structure & Concept Flow

Images cached at edge locations worldwide:

- User in Tokyo → Tokyo edge: 20ms latency

- User in Sydney → Sydney edge: 15ms latency

- User in London → London edge: 10ms latency

- Only cache misses hit origin (20% of requests)

SLIDE 44B: CDN Architecture - How It Works

How CDN Works:

CODE

#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .error-icon{fill:#552222;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .error-text{fill:#552222;stroke:#552222;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-thickness-normal{stroke-width:1px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-thickness-thick{stroke-width:3.5px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-pattern-solid{stroke-dasharray:0;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .marker{fill:#666;stroke:#666;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .marker.cross{stroke:#666;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d p{margin:0;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .cluster-label text{fill:#333;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .cluster-label span{color:#333;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .cluster-label span p{background-color:transparent;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .label text,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d span{fill:#000000;color:#000000;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node rect,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node circle,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node ellipse,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node polygon,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .rough-node .label text,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node .label text,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .image-shape .label,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .icon-shape .label{text-anchor:middle;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .rough-node .label,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node .label,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .image-shape .label,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .icon-shape .label{text-align:center;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node.clickable{cursor:pointer;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .arrowheadPath{fill:#333333;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .flowchart-link{stroke:#666;fill:none;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edgeLabel{background-color:white;text-align:center;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edgeLabel p{background-color:white;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .cluster text{fill:#333;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .cluster span{color:#333;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d rect.text{fill:none;stroke-width:0;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .icon-shape,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .image-shape{background-color:white;text-align:center;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .icon-shape p,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .image-shape p{background-color:white;padding:2px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .icon-shape .label rect,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d .node .neo-node{stroke:#999;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node rect,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].cluster rect,#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node polygon{stroke:url(#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node path{stroke:url(#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d-gradient);stroke-width:1px;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node circle{stroke:url(#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].icon-shape .icon{fill:url(#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-a8502f21-52e2-4f15-8d16-465df41ad83d :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}User in TokyoUser in SydneyUser in LondonCDN EdgeTokyoCDN EdgeSydneyCDN EdgeLondonOriginS3 us-east-1Another Userin Tokyo1. Request image2. Cache miss3. Return image4. Cache + serve5. Request same image6. Serve from cacheCache missCache miss

SLIDE 44C: CDN Architecture - Request Flow

CDN Request Flow:

First Request (Cache Miss):

Structure & Concept Flow

1. User requests: https://cdn.instagram.com/posts/789456_large.jpg

2. DNS resolves to nearest CDN edge (Tokyo for user in Tokyo)

3. Edge server checks local cache

4. Cache miss → Edge requests from origin (S3)

5. Edge receives image (2MB, takes 500ms from origin)

6. Edge caches image locally

7. Edge serves image to user (total: 520ms)

Subsequent Requests (Cache Hit):

Structure & Concept Flow

1. Another user in Tokyo requests same image

2. DNS resolves to Tokyo edge

3. Edge server checks local cache

4. Cache hit → Serve immediately

5. User receives image (total: 20ms)

SLIDE 44D: CDN Architecture - Metrics

Cache Hit Ratio:

Metric	Target	Typical
Cache Hit Ratio	80%	75-85%
Latency (Hit)	<50ms	20-40ms
Latency (Miss)	<500ms	300-600ms
Origin Bandwidth Saved	80%	75-85%

SLIDE 44E: CDN Architecture - Configuration

CDN Configuration:

Cache TTL (Time to Live):

CODE

Cache-Control: public, max-age=31536000, immutable

Explanation:
- public: Can be cached by CDN and browsers
- max-age=31536000: Cache for 1 year (365 days)
- immutable: File will never change (safe to cache forever)

Why 1 year?
- Images never change (if user edits, new URL with new post_id)
- Maximizes cache hit ratio
- Reduces origin load

Cache Invalidation:

Structure & Concept Flow

When needed (rarely):

- User deletes post → Purge from CDN

- User updates post image → New URL (new post_id), old cached version OK

API call to CDN:

POST /purge

{

"urls": [

"https://cdn.instagram.com/posts/789456_large.jpg",

"https://cdn.instagram.com/posts/789456_medium.jpg"

]

}

Propagation time: 5-10 minutes to all edge locations

SLIDE 44F: CDN Architecture - Geographic Distribution

Geographic Distribution:

Region	Edge Locations	Users Served
North America	50	200M
Europe	40	180M
Asia-Pacific	60	300M
South America	20	80M
Middle East / Africa	30	100M
Total	200 edge locations	860M

SLIDE 44G: CDN Architecture - Cost Analysis

Cost Analysis:

Without CDN (Serving from S3 Origin):

CODE

Egress bandwidth: 231 GB/sec × 86,400 sec/day × 30 days/month
= 600,000 TB/month

S3 egress cost: $0.09/GB = $90/TB
Monthly cost: 600,000 TB × $90 = $54,000,000/month

With CDN:

CODE

Origin egress (20% of traffic): 120,000 TB/month
S3 egress cost: 120,000 × $90 = $10,800,000

CDN egress (80% from cache): 480,000 TB/month
CDN cost: $0.085/GB = $85/TB
CDN cost: 480,000 × $85 = $40,800,000

Total: $51,600,000/month
Savings: $2,400,000/month (4.4% reduction)

But more importantly:
- Users get 10x faster load times
- Origin servers not overwhelmed
- System can actually handle the load

Note: In practice, CDN providers offer volume discounts (committed use), actual costs are 50-70% lower than list prices.

SLIDE 44H: CDN Architecture - Providers

CDN Providers:

Provider	Instagram’s Choice	Notes
Akamai	Used by Instagram	Largest CDN, most edge locations
CloudFront	AWS-native option	Good if already using AWS
Azure CDN	Microsoft option	Good if using Azure
Cloudflare	Popular choice	Good performance, DDoS protection

SLIDE 45: Media Storage Summary

Media Storage: Key Takeaways

Architecture Decisions:

Decision	Rationale
Object Storage (S3/Blob)	Designed for petabyte scale, 11-nines durability
Async Processing	Don’t block user during 10-second thumbnail generation
Multiple Thumbnail Sizes	Optimize bandwidth for different devices/networks
CDN for Delivery	Mandatory at 231 GB/sec egress, 10x latency improvement
Lifecycle Policies	Move old photos to cheaper storage tiers, save millions
Date-Based Partitioning	Easier management, better S3 performance

SLIDE 45B: Media Storage Summary - Storage Breakdown

Storage Breakdown:

CODE

Per Photo (4 files):
- Original: 2MB
- Large thumbnail: 500KB
- Medium thumbnail: 150KB
- Small thumbnail: 20KB
Total per photo: 2.67MB

Annual storage: 100M photos/day × 2.67MB × 365 days = 97.5 PB/year

SLIDE 45C: Media Storage Summary - Cost Optimization

Cost Optimization:

Structure & Concept Flow

5-Year Projection with Lifecycle Policies:

Year 1: Hot tier → $1.7M/month

Year 2: Moving to IA → $1.2M/month

Year 3: Moving to Glacier → $0.8M/month

Year 4: Mostly Glacier → $0.5M/month

Year 5: Mostly Glacier → $0.5M/month

Without lifecycle policies: $1.7M/month constant = $102M over 5 years

With lifecycle policies: ~$60M over 5 years

Savings: $42M (41% reduction)

SLIDE 45D: Media Storage Summary - Common Mistakes

Common Interview Mistakes:

Mistake	Correct Approach
Storing images in database	Use object storage, store URLs in database
Synchronous image processing	Async queue-based processing
No CDN	CDN mandatory for global delivery
Serving original images	Generate thumbnails, serve appropriate size
No lifecycle policies	Implement tiered storage for cost savings
Single storage region	Multi-region for disaster recovery

SLIDE 45E: Media Storage Summary - Interview Talking Points

Interview Talking Points:

“For media storage, I’m using object storage like S3 because it’s designed for petabyte scale and offers 11-nines durability. Images are processed asynchronously to avoid blocking the user - we generate multiple thumbnail sizes for different use cases. CDN is mandatory given the 231 GB/sec egress bandwidth requirement - it reduces latency by 10x and origin load by 80%. For cost optimization, we use lifecycle policies to automatically move old photos to cheaper storage tiers, saving millions annually.”