Chapter 2: CAPACITY ESTIMATION & ARCHITECTURE - System Design Course: Complete Interview Preparation | PlayBook Library

SLIDE 10: Why Capacity Estimation Matters

Step 2: Capacity Estimation

Why Interviewers Ask For This:

Tests Multiple Skills:

Ability to reason about scale and magnitude
Back-of-the-envelope calculation skills
Understanding of how scale impacts architecture
Practical engineering judgment

SLIDE 10B: Why Capacity Estimation Matters (continued)

Informs Architecture Decisions:

Storage requirements determine database sharding needs
QPS (Queries Per Second) determines number of servers needed
Bandwidth requirements determine CDN necessity
Cache size requirements determine Redis cluster sizing

What to Estimate:

Component	What to Calculate
Storage	Total data storage needs (photos, metadata)
Bandwidth	Network ingress (uploads) and egress (views)
QPS	Read and write queries per second
Cache	Redis cache size for hot data

SLIDE 10C: Why Capacity Estimation Matters (continued)

Interview Approach:

You don’t need exact numbers
Round generously (86,400 seconds ≈ 100,000 is acceptable)
Show your calculation process clearly
Explain assumptions explicitly

Common Mistake:
Skipping capacity estimation entirely or hand-waving with “we’ll scale horizontally.” Interviewers want to see you can reason about numbers.

SLIDE 11: Traffic Estimates - Assumptions

Traffic and QPS Calculations

Assumptions:

Parameter	Value	Reasoning
Daily Active Users (DAU)	500 million	Given in requirements
Average Posts Viewed per User	20 posts/day	Typical user engagement
Average Posts Created per User	0.2 posts/day	Not everyone posts daily

SLIDE 11B: Traffic Estimates - Read Traffic

Read Traffic (Feed Views):

CODE

Total daily views = 500M users × 20 posts/day
                 = 10 billion views/day

QPS = 10 billion / 86,400 seconds
    ≈ 10B / 100K (rounding for simplicity)
    ≈ 100,000 QPS

Peak traffic (assume 3x average) = 300,000 QPS

SLIDE 11C: Traffic Estimates - Write Traffic

Write Traffic (Photo Uploads):

CODE

Total daily uploads = 500M users × 0.2 posts/day
                    = 100 million posts/day

QPS = 100M / 86,400 seconds
    ≈ 100M / 100K
    ≈ 1,000 QPS

Peak traffic = 3,000 QPS

SLIDE 11D: Traffic Estimates - Read:Write Ratio

Read:Write Ratio:

CODE

Read:Write = 100,000 : 1,000
          = 100:1

This is a heavily read-skewed system.

Implications for Architecture:

Read optimization is critical (caching, CDN, read replicas)
Write path can be simpler (lower volume)
Database should be optimized for reads

SLIDE 12: Storage Estimates - Photo Storage

Storage Requirements

Photo Storage Calculations:

CODE

Assumptions:
- 100 million posts/day
- Average photo size: 2MB (after compression)
- Average video size: 10MB (we'll focus on photos primarily)

Daily photo storage:
= 100M posts × 2MB
= 200 TB/day

Annual photo storage:
= 200 TB × 365 days
= 73,000 TB/year
= 73 PB/year

SLIDE 12B: Storage Estimates - Metadata Storage

Metadata Storage (Database):

CODE

Assumptions:
- Post metadata: ~1KB per post (text, IDs, timestamps)
- User metadata: ~2KB per user
- Follows relationship: ~16 bytes per relationship
- Likes: ~16 bytes per like
- Comments: ~500 bytes per comment (average)

Daily metadata growth:
= 100M posts × 1KB (posts)
+ (100M posts × 10 likes/post × 16 bytes) (likes)
+ (100M posts × 2 comments/post × 500 bytes) (comments)
≈ 100 GB/day

Annual metadata growth:
≈ 36 TB/year

SLIDE 12C: Storage Estimates - Comparison

Storage Comparison:

Component	Annual Storage	Percentage
Media (Photos/Videos)	73 PB	99.95%
Metadata (Database)	36 TB	0.05%

Key Insight:
Media storage completely dominates. Architecture must optimize for media delivery (CDN is non-negotiable), while database storage is negligible in comparison.

SLIDE 13: Bandwidth Estimates - Ingress

Network Bandwidth Requirements

Ingress Bandwidth (Uploads):

CODE

Daily uploads: 100M posts × 2MB = 200 TB/day

Bandwidth = 200 TB / 86,400 seconds
          ≈ 200 TB / 100K seconds
          ≈ 2 GB/second ingress

SLIDE 13B: Bandwidth Estimates - Egress

Egress Bandwidth (Views):

CODE

Daily views: 10B views × 2MB per image = 20,000 TB/day

Bandwidth = 20,000 TB / 86,400 seconds
          ≈ 231 GB/second egress

Bandwidth Comparison:

Direction	Bandwidth	Ratio
Ingress (Uploads)	2.3 GB/sec	1x
Egress (Views)	231 GB/sec	100x

SLIDE 13C: Bandwidth Estimates - Critical Insight

Critical Insight:
Egress is 100x larger than ingress. This makes CDN (Content Delivery Network) absolutely mandatory. Origin servers cannot handle 231 GB/sec egress.

Cache Requirements (80/20 Rule):

CODE

Assumption: 20% of content receives 80% of views

Hot content to cache:
= 20% × 73 PB (annual storage)
= 14.6 PB of "hot" content

Active cache size needed:
= Recent content (last 7 days typically)
= 200 TB/day × 7 days
= 1.4 PB actively cached

SLIDE 13D: Bandwidth Estimates - Implications

Implications:

CDN is not optional at this scale
Aggressive caching reduces origin server load by 80%+
Cost of not using CDN would be prohibitive

SLIDE 14: High-Level Architecture Overview

High-Level Architecture: The 10,000-Foot View

CODE

#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .error-icon{fill:#552222;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .error-text{fill:#552222;stroke:#552222;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-thickness-normal{stroke-width:1px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .marker{fill:#666;stroke:#666;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .marker.cross{stroke:#666;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 p{margin:0;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .cluster-label text{fill:#333;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .cluster-label span{color:#333;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .cluster-label span p{background-color:transparent;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .label text,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 span{fill:#000000;color:#000000;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node rect,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node circle,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node ellipse,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node polygon,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .rough-node .label text,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node .label text,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .image-shape .label,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .icon-shape .label{text-anchor:middle;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .rough-node .label,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node .label,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .image-shape .label,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .icon-shape .label{text-align:center;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node.clickable{cursor:pointer;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .arrowheadPath{fill:#333333;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .flowchart-link{stroke:#666;fill:none;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edgeLabel{background-color:white;text-align:center;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edgeLabel p{background-color:white;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .cluster text{fill:#333;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .cluster span{color:#333;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 rect.text{fill:none;stroke-width:0;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .icon-shape,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .image-shape{background-color:white;text-align:center;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .icon-shape p,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .image-shape p{background-color:white;padding:2px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .icon-shape .label rect,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 .node .neo-node{stroke:#999;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node rect,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].cluster rect,#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node polygon{stroke:url(#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node path{stroke:url(#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9-gradient);stroke-width:1px;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node circle{stroke:url(#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-fa038983-3354-439f-8f41-bb0ef12662b9 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}Data LayerApplication LayerEdge LayerClient LayerUser DBSharded PostgreSQLPost DBSharded PostgreSQLFeed CacheRedis ClusterObject StorageS3 / Azure BlobAPI GatewayAuth, Rate Limiting, RoutingUser ServiceProfiles, AuthPost ServiceCreate, Delete PostsFeed ServiceGenerate FeedsMedia ServiceUpload, ProcessLoad BalancerLayer 7 HTTPCDNCloudFront / Azure CDNMobile AppsiOS, AndroidWeb Browsers

SLIDE 14B: High-Level Architecture - Principles

Architecture Principles:

Separation of Concerns:

Each service has a single, well-defined responsibility
Services own their data (no shared databases between services)
Independent scaling per service

Layers:

Client Layer: Mobile apps and web browsers
Edge Layer: Load balancing and CDN for media delivery
Application Layer: Microservices handling business logic
Data Layer: Databases, caches, and object storage

SLIDE 14C: High-Level Architecture - Key Decisions

Key Design Decisions:

Microservices architecture (not monolith)
Separate media delivery path (CDN)
Centralized API Gateway for cross-cutting concerns
Dedicated caching layer for feed performance

SLIDE 15: Core Components - API Gateway

Component Breakdown

1. API Gateway

Responsibilities:

Single entry point for all client requests
Authentication and authorization (JWT token validation)
Rate limiting per user/IP
Request routing to appropriate microservice
API versioning support

Technology:

Kong, AWS API Gateway, or Azure API Management

SLIDE 15B: Core Components - User Service

2. User Service

Responsibilities:

User registration and authentication
User profile management (bio, profile picture, etc.)
Follow/unfollow relationships
User search functionality
Friend suggestions (out of scope for MVP)

Data Owned:

User profiles
Follow graph (who follows whom)

SLIDE 15C: Core Components - Post Service

3. Post Service

Responsibilities:

Create new posts (coordinates with Media Service)
Delete posts
Retrieve posts by user
Manage post metadata (captions, location, timestamps)
Like/unlike functionality
Comment management

Data Owned:

Post metadata
Likes
Comments

SLIDE 15D: Core Components - Feed Service

4. Feed Service

Responsibilities:

Generate personalized feed for each user
Rank and sort posts by relevance
Handle pagination
Cache feed results
Most complex service (detailed later)

Data Owned:

Feed cache (pre-computed feeds)

SLIDE 15E: Core Components - Media Service

5. Media Service

Responsibilities:

Handle image/video uploads
Generate multiple thumbnail sizes
Store media in object storage
Coordinate with CDN
Image compression and optimization

Data Owned:

References to media in object storage

Service Independence:
Each service can be developed, deployed, and scaled independently by separate teams.

SLIDE 16: Microservices Justification - Benefits

Architectural Decision: Microservices vs Monolith

Why Microservices for Instagram:

Benefits:

Benefit	Explanation	Example
Independent Scaling	Scale services based on their specific load	Feed Service needs 100x more instances than Post Service
Team Autonomy	Different teams own different services	Feed team can deploy without coordinating with Media team

SLIDE 16B: Microservices Justification - Benefits (continued)

Benefit	Explanation	Example
Fault Isolation	One service failure doesn’t crash entire system	Media Service down doesn’t prevent feed viewing
Technology Flexibility	Each service can use optimal tech stack	Feed Service in Java, Media Service in Go for performance
Deployment Flexibility	Deploy services independently	Fix bug in User Service without redeploying everything

SLIDE 16C: Microservices Justification - Trade-Offs

Trade-Offs (Microservices Complexity):

Drawback	Mitigation Strategy
Network overhead between services	Use efficient protocols (gRPC), co-locate related services
Distributed transaction challenges	Use eventual consistency, saga pattern for transactions
Operational complexity	Invest in monitoring, logging, distributed tracing
More complex deployment	Use container orchestration (Kubernetes)

SLIDE 16D: Microservices Justification - Monolith Alternative

Alternative: Monolith

When Monolith Makes Sense:

Early-stage startup with small team
Unclear product-market fit
Simpler deployment and debugging

Why Not Monolith for Instagram:

Scale requirements exceed single-server capacity
Multiple teams need to work independently
Different components have different scaling needs

Recommendation:
At Instagram’s scale (500M DAU), microservices benefits outweigh complexity costs. In production systems handling billions of requests, independent scaling and fault isolation are critical.

SLIDE 17: Data Flow - Upload Path (Diagram)

Data Flow: User Uploads a Photo

Structure & Concept Flow

CDNPost DatabasePost ServiceObject StorageImage WorkerProcessing QueueMedia ServiceAPI GatewayClientCDNPost DatabasePost ServiceObject StorageImage WorkerProcessing QueueMedia ServiceAPI GatewayClient#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .error-icon{fill:#552222;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .error-text{fill:#552222;stroke:#552222;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-thickness-normal{stroke-width:1px;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .marker{fill:#666;stroke:#666;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .marker.cross{stroke:#666;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 p{margin:0;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .actor{stroke:hsl(0, 0%, 83%);fill:#eee;stroke-width:1;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 rect.actor.outer-path[data-look="neo"]{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 rect.note[data-look="neo"]{stroke:#999;fill:#666;filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 text.actor>tspan{fill:#333;stroke:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .actor-line{stroke:hsl(0, 0%, 83%);}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [id$="-arrowhead"] path{fill:#333;stroke:#333;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .sequenceNumber{fill:white;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [id$="-sequencenumber"]{fill:#333;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [id$="-crosshead"] path{fill:#333;stroke:#333;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .messageText{fill:#333;stroke:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .labelBox{stroke:hsl(0, 0%, 83%);fill:#eee;filter:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .labelText,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .labelText>tspan{fill:#333;stroke:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .loopText,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .loopText>tspan{fill:#333;stroke:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .sectionTitle,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .sectionTitle>tspan{fill:#333;stroke:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(0, 0%, 83%);fill:hsl(0, 0%, 83%);}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .note{stroke:#999;fill:#666;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .noteText,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .noteText>tspan{fill:#fff;stroke:none;font-weight:normal;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .actorPopupMenu{position:absolute;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .actorPopupMenuPanel{position:absolute;fill:#eee;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .actor-man circle,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 line{fill:#eee;stroke-width:2px;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 g rect.rect{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));stroke:#999;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 .node .neo-node{stroke:#999;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node rect,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].cluster rect,#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node polygon{stroke:url(#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node path{stroke:url(#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9-gradient);stroke-width:1px;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node circle{stroke:url(#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-0b55530a-3304-45fe-86f3-427558b77ca9 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}POST /upload(image file, caption, location)Validate auth tokenForward upload requestEnqueue processing job202 Accepted(job_id: abc123)Pick up jobCompress image(5MB → 2MB)Generate thumbnails(small, medium, large)Store original + thumbnailsURLsCreate post metadata(image_urls, caption)INSERT INTO postspost_id: 789456Invalidate cache (if needed)WebSocket notification"Post created successfully"

SLIDE 17B: Data Flow - Upload Path (Steps 1-2)

Upload Flow Steps:

1. Client Initiates Upload (1-2 seconds)

User selects photo in app
App sends POST request with image file, caption, and location
API Gateway validates authentication token

2. Async Processing Begins (Immediate Response)

Media Service receives upload
Immediately enqueues job in processing queue
Returns 202 Accepted (job accepted, processing asynchronously)
Client shows “Uploading…” indicator

SLIDE 17C: Data Flow - Upload Path (Steps 3-4)

3. Background Worker Processes Image (5-10 seconds)

Worker picks job from queue
Compresses original image (5MB → 2MB JPEG, quality 85%)
Generates 3 thumbnail sizes:
- Small: 150×150 (for profile views)
- Medium: 640×640 (for feed display)
- Large: 1080×1080 (for detail view)

4. Storage and Metadata Creation

Stores 4 files in S3 (original + 3 thumbnails)
Creates post metadata in Post Service database
Associates image URLs with post

SLIDE 17D: Data Flow - Upload Path (Step 5 and Why Async)

5. Client Notification

WebSocket notification sent to client
Client UI updates to show post successfully created

Why Async Processing:

User doesn’t wait 10 seconds for upload to complete
Image processing is CPU-intensive (don’t block API servers)
Handles traffic spikes (queue buffers requests)

SLIDE 18: Data Flow - Feed Load Path (Diagram)

Data Flow: User Loads Their Feed

CODE

CDNUser DatabasePost DatabaseFeed CacheFeed ServiceAPI GatewayClientCDNUser DatabasePost DatabaseFeed CacheFeed ServiceAPI GatewayClient#mermaid-b2309e56-ddff-455c-9825-de4069f23748{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .error-icon{fill:#552222;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .error-text{fill:#552222;stroke:#552222;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-thickness-normal{stroke-width:1px;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .marker{fill:#666;stroke:#666;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .marker.cross{stroke:#666;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 p{margin:0;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .actor{stroke:hsl(0, 0%, 83%);fill:#eee;stroke-width:1;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 rect.actor.outer-path[data-look="neo"]{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 rect.note[data-look="neo"]{stroke:#999;fill:#666;filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 text.actor>tspan{fill:#333;stroke:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .actor-line{stroke:hsl(0, 0%, 83%);}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [id$="-arrowhead"] path{fill:#333;stroke:#333;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .sequenceNumber{fill:white;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [id$="-sequencenumber"]{fill:#333;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [id$="-crosshead"] path{fill:#333;stroke:#333;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .messageText{fill:#333;stroke:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .labelBox{stroke:hsl(0, 0%, 83%);fill:#eee;filter:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .labelText,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .labelText>tspan{fill:#333;stroke:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .loopText,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .loopText>tspan{fill:#333;stroke:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .sectionTitle,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .sectionTitle>tspan{fill:#333;stroke:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(0, 0%, 83%);fill:hsl(0, 0%, 83%);}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .note{stroke:#999;fill:#666;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .noteText,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .noteText>tspan{fill:#fff;stroke:none;font-weight:normal;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .actorPopupMenu{position:absolute;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .actorPopupMenuPanel{position:absolute;fill:#eee;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .actor-man circle,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 line{fill:#eee;stroke-width:2px;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 g rect.rect{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));stroke:#999;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 .node .neo-node{stroke:#999;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node rect,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].cluster rect,#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node polygon{stroke:url(#mermaid-b2309e56-ddff-455c-9825-de4069f23748-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node path{stroke:url(#mermaid-b2309e56-ddff-455c-9825-de4069f23748-gradient);stroke-width:1px;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node circle{stroke:url(#mermaid-b2309e56-ddff-455c-9825-de4069f23748-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-b2309e56-ddff-455c-9825-de4069f23748-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-b2309e56-ddff-455c-9825-de4069f23748-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-b2309e56-ddff-455c-9825-de4069f23748 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}alt[Cache Hit (80% of requests)][Cache Miss (20% of requests)]GET /feed?limit=20Validate auth tokenExtract user_idGet feed for user_id=12345GET feed:12345[post_id_1, post_id_2, ...]Feed cached, skip generationnil (not in cache)Get users followed by 12345[user_id: 999, 888, 777, ...]Get recent postsfrom [999, 888, 777][post_ids with metadata]Rank posts by algorithmSET feed:12345 (TTL: 5 min)Hydrate post detailsfor post_idsFull post objectsGet user info for postsUser profilesFeed response (JSON)200 OK (feed data)Request images (parallel)Images

SLIDE 18B: Data Flow - Feed Load Path (Steps 1-2)

Feed Load Steps:

1. Request Initiation

Client sends GET /feed request
API Gateway validates authentication
Extracts user_id from JWT token

2. Cache Check (Critical for Performance)

Feed Service checks Redis for pre-computed feed
Cache key: feed:{user_id}
Cache TTL: 5 minutes
80% cache hit rate expected

SLIDE 18C: Data Flow - Feed Load Path (Cache Hit)

3A. Cache Hit Path (Fast - 50-100ms total)

Retrieve cached list of post IDs
Skip feed generation entirely
Proceed to hydration

SLIDE 18D: Data Flow - Feed Load Path (Cache Miss)

3B. Cache Miss Path (Slower - 300-500ms total)

Query User Database: “Who does user 12345 follow?”
Query Post Database: “Recent posts from those users”
Apply ranking algorithm (recency, engagement, etc.)
Store result in Redis cache for 5 minutes

SLIDE 18E: Data Flow - Feed Load Path (Hydration and Response)

4. Hydration (Both Paths)

Convert post IDs to full post objects
Fetch user information for each post author
Calculate aggregated counts (likes, comments)
Check if current user has liked each post

5. Response

Return JSON feed to client
Client requests images in parallel from CDN
CDN serves images (cached at edge locations)

SLIDE 18F: Data Flow - Feed Load Path (Performance)

Performance Breakdown:

Step	Cache Hit	Cache Miss
Cache Check	10ms	10ms
Feed Generation	0ms	200-300ms
Hydration	50ms	50ms
Total	60-100ms	260-360ms

SLIDE 19: Architecture Summary

High-Level Architecture: Key Takeaways

Architectural Patterns Used:

1. Microservices Architecture

Independent services for user, post, feed, and media management
Each service scales independently
Services communicate via REST APIs or message queues

2. Separation of Media from Metadata

Media files stored in object storage (S3/Blob)
Metadata stored in relational database (PostgreSQL)
This separation is critical for scale

SLIDE 19B: Architecture Summary (continued)

3. Multi-Layer Caching

CDN for media files (80%+ hit rate)
Redis for feed and user data (5-minute TTL)
Database query cache

4. Async Processing

Image processing happens asynchronously
Feed generation can be pre-computed
Non-blocking user experience

5. Read-Optimized Design

Read replicas for database
Aggressive caching at all layers
CDN for 100x read traffic

SLIDE 19C: Architecture Summary - Key Decisions

What to Emphasize in Interview:

Decision	Rationale
Microservices	Independent scaling, fault isolation, team autonomy
CDN	Mandatory at 231 GB/sec egress bandwidth
Redis Caching	Feed generation is expensive, caching critical
Object Storage	Designed for petabyte-scale media storage
Async Processing	User doesn’t wait for thumbnail generation

Next Section:
Now that we have high-level architecture, let’s dive deep into database design - the foundation of our metadata storage.