Chapter 2: Servers and Application Architecture - System Design Course: Complete Foundations for Interviews | PlayBook Library

SLIDE 15: What is a Server

Section 3: Servers and Application Architecture

A server is a computer that listens for incoming requests, processes them, and sends back responses. There is nothing special about the hardware. Any laptop can be a server. What makes a machine a “server” is the software running on it and the role it plays in the architecture.

What We’ll Cover:

Single server architecture
Stateful vs stateless servers
Server types (web, application, database)
Reverse proxies
API gateways
Monolith vs microservices

Physical vs Cloud Servers:

Type	Description	Example
Bare metal	Physical machine in a data center, you own the hardware	Dell PowerEdge, HP ProLiant
Virtual machine (VM)	Software-defined server running on shared hardware	AWS EC2, Azure VM, Google Compute Engine
Container	Lightweight, isolated process sharing the host OS kernel	Docker container running on Kubernetes
Serverless	No server to manage, cloud provider runs your code on demand	AWS Lambda, Azure Functions

Typical Production Server Specs:

CODE

A mid-range cloud server (e.g., AWS m5.2xlarge):
- CPU: 8 vCPUs
- RAM: 32 GB
- Storage: 500 GB SSD (EBS)
- Network: Up to 10 Gbps
- Cost: ~$280/month (on-demand), ~$170/month (reserved 1-year)

This single machine can handle:
- 1,000-5,000 concurrent connections
- 500-2,000 HTTP requests per second (depending on complexity)
- 10-50 database queries per request

At YouTube’s scale (500,000+ requests per second), you need hundreds to thousands of these machines working together. The rest of this section covers how to organize them.

SLIDE 16: Single Server Architecture

Single Server Architecture

The simplest possible system runs everything on one machine: the web server, the application code, and the database.

CODE

#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;fill:#000000;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .error-icon{fill:#552222;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .error-text{fill:#552222;stroke:#552222;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-thickness-normal{stroke-width:1px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .marker{fill:#666;stroke:#666;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .marker.cross{stroke:#666;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 svg{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:16px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 p{margin:0;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .label{font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";color:#000000;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .cluster-label text{fill:#333;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .cluster-label span{color:#333;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .cluster-label span p{background-color:transparent;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .label text,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 span{fill:#000000;color:#000000;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node rect,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node circle,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node ellipse,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node polygon,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node path{fill:#eee;stroke:#999;stroke-width:1px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .rough-node .label text,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node .label text,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .image-shape .label,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .icon-shape .label{text-anchor:middle;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .rough-node .label,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node .label,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .image-shape .label,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .icon-shape .label{text-align:center;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node.clickable{cursor:pointer;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .root .anchor path{fill:#666!important;stroke-width:0;stroke:#666;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .arrowheadPath{fill:#333333;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edgePath .path{stroke:#666;stroke-width:1px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .flowchart-link{stroke:#666;fill:none;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edgeLabel{background-color:white;text-align:center;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edgeLabel p{background-color:white;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .edgeLabel rect{opacity:0.5;background-color:white;fill:white;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .cluster rect{fill:hsl(0, 0%, 98.9215686275%);stroke:#707070;stroke-width:1px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .cluster text{fill:#333;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .cluster span{color:#333;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:ui-sans-serif,-apple-system,BlinkMacSystemFont,"Segoe UI Variable Display","Segoe UI",Helvetica,"Apple Color Emoji","Noto Sans Arabic","Noto Sans Hebrew",Arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol";font-size:12px;background:hsl(-160, 0%, 93.3333333333%);border:1px solid #707070;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#000000;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 rect.text{fill:none;stroke-width:0;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .icon-shape,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .image-shape{background-color:white;text-align:center;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .icon-shape p,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .image-shape p{background-color:white;padding:2px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .icon-shape .label rect,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .image-shape .label rect{opacity:0.5;background-color:white;fill:white;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 .node .neo-node{stroke:#999;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node rect,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].cluster rect,#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node polygon{stroke:url(#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].swimlane.cluster rect{filter:none;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node path{stroke:url(#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6-gradient);stroke-width:1px;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node .neo-line path{stroke:#999;filter:none;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node circle{stroke:url(#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].node circle .state-start{fill:#000000;}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].icon-shape .icon{fill:url(#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 [data-look="neo"].icon-shape .icon-neo path{stroke:url(#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#mermaid-5b397e60-1e3d-4223-931d-3c343e7fe3b6 :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}Client 1BrowserClient 2Mobile AppClient 3BrowserSingle ServerWeb Server + App Code + DatabaseIP: 203.0.113.50

What Runs on This Single Server:

CODE

1. Web Server (Nginx or Apache)
   - Listens on port 80/443
   - Serves static files (HTML, CSS, JS, images)
   - Forwards dynamic requests to application code

2. Application Code (Node.js, Django, Spring Boot, etc.)
   - Business logic
   - User authentication
   - API endpoints
   - Runs on port 3000/8080 (internal)

3. Database (PostgreSQL, MySQL)
   - Stores all persistent data
   - Runs on port 5432/3306 (internal)
   - Data stored on local disk

When This Works:

Personal projects, prototypes, early MVPs
Low traffic (under 1,000 daily active users)
Small dataset (under 10 GB)
Single developer, no team coordination needed

When This Breaks:

Problem	Symptom	Threshold
CPU exhaustion	Requests queue up, response times spike	>70% sustained CPU usage
Memory exhaustion	Process crashes, OOM (Out of Memory) kills	>90% RAM usage
Disk I/O bottleneck	Database queries slow to seconds	Disk queue depth > 2
Network saturation	Packet loss, timeouts	>80% bandwidth utilization
Single point of failure	Server crashes = entire system down	Any hardware failure

The first scaling step is always the same: separate the database from the application server. This is covered in Section 5 (Scaling).

SLIDE 17: Stateful vs Stateless Servers

Stateful vs Stateless Servers

This distinction is one of the most important concepts in system design. It determines whether your application can scale horizontally.

Stateful Server:

A stateful server stores client-specific data (like session information) in its own memory. The server “remembers” each client between requests.

Structure & Concept Flow

Example: User logs in to Server A

Request 1 → Server A

Server A stores: session = {user_id: 123, cart: [item1, item2]}

Server A memory now holds this user's state

Request 2 → Server A

Server A finds session in its memory → works correctly ✓

Request 2 → Server B (different server)

Server B has no memory of this user → session not found ✗

User appears logged out, shopping cart is empty

Stateless Server:

A stateless server stores no client-specific data in its own memory. All state lives in external shared storage (database, cache, or the client itself via tokens).

Structure & Concept Flow

Example: User logs in, receives a JWT token

Request 1 → Server A

Client sends JWT token in header

Server A validates token, queries Redis for cart → works ✓

Request 2 → Server B (different server)

Client sends same JWT token in header

Server B validates token, queries same Redis for cart → works ✓

Any server can handle any request.

SLIDE 17B: Stateful vs Stateless: Implications for Scaling

Why Stateless Matters for Scaling:

Property	Stateful	Stateless
Horizontal scaling	Difficult (session affinity required)	Simple (add more servers)
Load balancing	Must route user to same server (sticky sessions)	Any server can handle any request
Server failure	User loses session, must re-login	No impact, next request goes to another server
Deployment	Rolling restart loses active sessions	Rolling restart is seamless
Memory usage	Grows with number of active users	Constant (no per-user memory)

Where State Goes Instead:

Structure & Concept Flow

In a stateless architecture, state moves to external stores:

1. Session data → Redis or Memcached

Key: "session:abc123"

Value: {user_id: 123, cart: [item1, item2], login_time: ...}

TTL: 30 minutes

2. Authentication → JWT tokens (stored on client)

Token contains: user_id, roles, expiration

Server validates token cryptographically, no lookup needed

3. User data → Database (PostgreSQL, MySQL)

Persistent storage, survives server restarts

4. Temporary data → Message queue (Kafka, SQS)

Async jobs, event processing

Stateless Architecture Diagram:

Interview Tip:
When you propose adding servers in a design, always mention that your application servers are stateless. If the interviewer asks “what happens if Server B goes down?”, the answer should be “nothing visible to the user, the load balancer routes to Server A or C, and they can serve the request identically because all state is in Redis/database.”

SLIDE 18: Web Servers, Application Servers, and Database Servers

Server Types

In production, different types of servers handle different responsibilities. Separating these allows each to be optimized and scaled independently.

1. Web Server (Nginx, Apache, Caddy)

CODE

Role: Handle HTTP connections, serve static files, forward dynamic requests

Responsibilities:
- Accept incoming TCP/HTTP connections
- Serve static files directly (HTML, CSS, JS, images)
- Forward dynamic requests to application server (reverse proxy)
- TLS termination (decrypt HTTPS, forward HTTP internally)
- Compression (gzip/brotli for text responses)
- Connection management (keep-alive, timeouts)

Performance:
- Nginx can handle 10,000+ concurrent connections
- Serving a static file: ~1ms
- Forwarding to application server: adds ~1-2ms overhead

2. Application Server (Node.js, Django, Spring Boot, ASP.NET)

CODE

Role: Execute business logic, process requests, return responses

Responsibilities:
- Route requests to correct handler
- Validate input data
- Execute business logic (compute feed, process payment, etc.)
- Query database or cache
- Build response (JSON, HTML)
- Handle authentication and authorization

Performance:
- Typical API response: 10-200ms
- CPU-bound work (image processing, ML inference): 100ms-10s
- I/O-bound work (database queries): 5-50ms per query

3. Database Server (PostgreSQL, MySQL, MongoDB, Redis)

CODE

Role: Store, retrieve, and manage persistent data

Responsibilities:
- Execute queries (SELECT, INSERT, UPDATE, DELETE)
- Maintain data integrity (ACID properties, constraints)
- Manage indexes for query performance
- Handle concurrent access (locking, transactions)
- Replication and backup

Performance:
- Simple key lookup (indexed): <1ms
- Complex join query: 10-100ms
- Full table scan (no index): 100ms-10s (depending on table size)

How They Work Together:

CODE

Client request: GET /api/users/123

1. Web Server (Nginx) receives HTTPS request on port 443
2. Decrypts TLS, forwards HTTP to Application Server on port 8080
3. Application Server validates JWT token, calls database
4. Database Server returns user data
5. Application Server formats JSON response
6. Web Server sends response back to client over HTTPS

SLIDE 19: Reverse Proxies

Reverse Proxy

A reverse proxy sits between clients and your backend servers. The client sends requests to the reverse proxy, and the proxy forwards them to the appropriate backend server. The client never communicates directly with backend servers.

What a Reverse Proxy Does:

Function	Explanation
Load distribution	Distributes requests across multiple backend servers
TLS termination	Handles encryption/decryption so backends don’t have to
Compression	Compresses responses (gzip, brotli) before sending to client
Caching	Caches static responses, reducing backend load
Rate limiting	Limits requests per IP or per user
Security	Hides backend server IPs from clients, blocks malicious requests

Forward Proxy vs Reverse Proxy:

Structure & Concept Flow

Forward proxy (client-side):

Client → Forward Proxy → Internet → Server

Example: Corporate proxy, VPN

Protects/hides the CLIENT

Reverse proxy (server-side):

Client → Internet → Reverse Proxy → Backend Servers

Example: Nginx in front of application servers

Protects/hides the SERVER

Common Reverse Proxy Technologies:

Technology	Use Case
Nginx	Most popular, high performance, low memory
HAProxy	Advanced load balancing, TCP and HTTP
Envoy	Microservices service mesh (Istio uses Envoy)
Traefik	Container-native, auto-discovers services
Cloudflare	CDN + reverse proxy as a service

In practice, Nginx is the default choice for most production systems. It can handle tens of thousands of concurrent connections with minimal memory overhead.

SLIDE 20: API Gateways

API Gateway

An API gateway is a specialized reverse proxy designed for API traffic. It adds application-level concerns on top of basic proxying.

Reverse Proxy vs API Gateway:

Feature	Reverse Proxy (Nginx)	API Gateway (Kong, AWS API Gateway)
Load balancing	Yes	Yes
TLS termination	Yes	Yes
Request routing	Basic (URL path matching)	Advanced (headers, query params, methods)
Authentication	No (or basic)	Yes (JWT validation, OAuth, API keys)
Rate limiting	Basic (per-IP)	Advanced (per-user, per-endpoint, per-plan)
Request transformation	No	Yes (modify headers, body, query params)
API versioning	No	Yes (route /v1/ and /v2/ to different services)
Analytics and logging	Basic access logs	Detailed API usage metrics
Circuit breaking	No	Yes (stop calling failing services)

API Gateway in a Microservices Architecture:

What Happens at the API Gateway:

CODE

1. Client sends: GET /api/feed?page=1
   Headers: Authorization: Bearer eyJhbG...

2. API Gateway:
   a. Validates JWT token (is it valid? is it expired?)
   b. Extracts user_id from token
   c. Checks rate limit (has this user exceeded 120 requests/hour?)
   d. Routes request to Feed Service based on URL path
   e. Adds internal headers: X-User-Id: 12345
   f. Logs: timestamp, user_id, endpoint, response_time

3. Feed Service receives the request with user_id already validated.
   It does not need to handle authentication itself.

Common API Gateway Technologies:

Technology	Type	Best For
Kong	Open source / Enterprise	Self-hosted, plugin ecosystem
AWS API Gateway	Managed service	AWS-native architectures
Azure API Management	Managed service	Azure-native architectures
Apigee (Google)	Managed service	GCP-native, enterprise
Traefik	Open source	Kubernetes-native

Interview Tip:
In system design interviews, always include an API gateway in your architecture diagram. It shows you understand cross-cutting concerns (auth, rate limiting, routing) and that you don’t want every microservice re-implementing these independently.

SLIDE 21: Monolith Architecture

Monolith Architecture

A monolith is a single deployable unit that contains all of the application’s functionality. All features, all business logic, all data access code lives in one codebase and deploys as one process.

Structure:

Structure & Concept Flow

my-application/

├── src/

│ ├── users/ # User registration, profiles, authentication

│ ├── posts/ # Post creation, deletion, retrieval

│ ├── feed/ # Feed generation, ranking

│ ├── media/ # Image upload, processing

│ ├── notifications/ # Push notifications, email

│ ├── search/ # User and content search

│ └── shared/ # Database connections, utilities, models

├── database/

│ └── schema.sql # Single database, all tables

└── deploy.sh # One deployment script, one server

How It Runs:

Structure & Concept Flow

One process, one server:

Server (port 8080):

/api/users/* → users module

/api/posts/* → posts module

/api/feed/* → feed module

/api/media/* → media module

All modules share:

- Same database connection pool

- Same memory space

- Same deployment lifecycle

- Same programming language

Advantages:

Advantage	Explanation
Simple to develop	One codebase, one IDE, one build process
Simple to deploy	Build one artifact, deploy to one server
Simple to debug	One process, one log stream, stack traces span full request
Simple to test	Integration tests run against one process
No network overhead	Modules call each other via function calls, not HTTP
Transaction support	Single database, ACID transactions across all tables

When Monolith is the Right Choice:

Team of 1-10 engineers
Early-stage product (uncertain requirements, rapid iteration)
Under 100K daily active users
Single deployment environment

Most successful companies started as monoliths: Shopify, GitHub, Stack Overflow, Basecamp. Some remain monoliths at significant scale.

SLIDE 22: Microservices Architecture

Microservices Architecture

A microservices architecture splits the application into small, independently deployable services. Each service owns a specific business capability and its own data.

Structure:

Structure & Concept Flow

user-service/ # Separate repo, separate team

├── src/

├── database/ # Own database (users, follows)

├── Dockerfile

└── deploy.yaml

post-service/ # Separate repo, separate team

├── src/

├── database/ # Own database (posts, likes, comments)

├── Dockerfile

└── deploy.yaml

feed-service/ # Separate repo, separate team

├── src/

├── cache/ # Own Redis cache (pre-computed feeds)

├── Dockerfile

└── deploy.yaml

media-service/ # Separate repo, separate team

├── src/

├── object-storage/ # S3 bucket (images, videos)

├── Dockerfile

└── deploy.yaml

How Services Communicate:

Communication Patterns:

Pattern	When Used	Example
Synchronous (REST/gRPC)	Need immediate response	Feed Service calls User Service to get profile info
Asynchronous (message queue)	Can be processed later	Post created event triggers feed regeneration
Event-driven (pub/sub)	Multiple consumers need to react	New post triggers feed update AND notification AND analytics

Key Principle: Each Service Owns Its Data

CODE

User Service:   Only service that reads/writes to Users database
Post Service:   Only service that reads/writes to Posts database
Feed Service:   Only service that reads/writes to Feed cache

If Feed Service needs user data, it calls User Service's API.
It does NOT query the Users database directly.

This is called "database per service" and is fundamental to
microservices. Without it, services are coupled at the data layer
and cannot be deployed or scaled independently.

SLIDE 22B: Monolith vs Microservices: Trade-Offs

Monolith vs Microservices: Trade-Off Comparison

Dimension	Monolith	Microservices
Development speed (early)	Faster (no service boundaries)	Slower (must define APIs between services)
Development speed (at scale)	Slower (large codebase, merge conflicts)	Faster (small, focused codebases)
Deployment	One deployment, all or nothing	Independent deployment per service
Scaling	Scale entire application	Scale individual services
Failure isolation	Bug in one module can crash everything	Bug in Feed Service doesn’t affect User Service
Data consistency	Easy (single database, ACID)	Hard (distributed transactions, eventual consistency)
Debugging	Simple (one process, one log)	Complex (distributed tracing across services)
Team structure	One team, shared codebase	Separate teams per service
Technology choice	One language, one framework	Each service can use different tech
Operational cost	Low (one server, one deployment)	High (many servers, orchestration, monitoring)

The Real-World Decision:

CODE

Instagram's actual history:
- 2010: Launched as a monolith (Django, Python)
- 2012: Acquired by Facebook (30M users)
- 2013-2016: Gradually decomposed into microservices
- 2024: Runs hundreds of microservices

The lesson:
Start monolith. Split into microservices only when you have
a specific scaling or organizational problem that requires it.

Premature microservices is one of the most common architectural mistakes.

When to Split a Monolith:

Signal	What It Looks Like
Team scaling pain	Merge conflicts daily, deploys blocked by other teams
Independent scaling needed	Feed generation needs 100 servers, user auth needs 5
Deployment frequency mismatch	Payment team ships weekly, feed team ships daily
Different reliability requirements	Media upload can be down, feed must always work
Technology mismatch	ML team needs Python, API team needs Java

SLIDE 23: Monolith vs Microservices: Selection Criteria

Decision Framework

Choose Monolith When:

Small team (under 10 engineers)
New product with unclear requirements
Simple domain (not many distinct business capabilities)
Speed of initial delivery matters most
You don’t yet know which components need independent scaling

Choose Microservices When:

Large team (multiple teams, 20+ engineers)
Well-understood domain with clear service boundaries
Components have very different scaling requirements
Teams need to deploy independently
Fault isolation is critical (one component failing must not bring down others)

Common Interview Mistakes:

CODE

Mistake 1: Defaulting to microservices for every design
"I'll use microservices because they're the industry standard."

Problem: Most interview questions don't require microservices.
A URL shortener serving 1M users doesn't need 5 separate services.

Better: "For this scale and complexity, a modular monolith would be
sufficient. I'd consider splitting into services if the user base
grows beyond 100M or if the team grows to multiple independent squads."

Mistake 2: Not acknowledging trade-offs
"Microservices are always better than monoliths."

Problem: Shows lack of experience. Every architecture has costs.

Better: "Microservices give us independent scaling and fault isolation,
but add complexity in service communication, data consistency, and
operational overhead. At this scale, the benefits outweigh those costs."

Mistake 3: Splitting by technical layer instead of business capability
Wrong: "Frontend service, Backend service, Database service"
Right: "User service, Order service, Payment service, Notification service"

Services should be organized by business domain, not by technical layer.
This aligns with Conway's Law: system architecture mirrors team structure.

SLIDE 24: Section Summary and Interview Tips

Section 3 Summary: Servers and Application Architecture

Concepts Covered:

Concept	One-Line Summary
Server types	Physical, VM, container, serverless (all are just computers running software)
Single server	Simplest architecture, works for <1,000 DAU
Stateful servers	Store session in memory, cannot scale horizontally
Stateless servers	Store state externally (Redis, DB), any server can handle any request
Web server	Handles HTTP, serves static files, terminates TLS (Nginx)
Application server	Executes business logic (Node.js, Django, Spring Boot)
Database server	Stores and retrieves persistent data (PostgreSQL, MySQL)
Reverse proxy	Sits in front of backends, handles TLS, load distribution, caching
API gateway	Reverse proxy with auth, rate limiting, routing, analytics
Monolith	Single deployable unit, all features in one codebase
Microservices	Independent services per business capability, own their data

How This Connects to the Rest of the Course:

CODE

Section 4 (Databases):     The data layer that application servers query
Section 5 (Scaling):       Adding more stateless servers behind a load balancer
Section 6 (Caching):       External state store (Redis) referenced in stateless design
Section 7 (Load Balancing): Distributing traffic across stateless servers
Section 9 (Queues):        Async communication between microservices
Section 10 (APIs):         Designing the HTTP endpoints on application servers

Key Interview Takeaways:

Always state that your application servers are stateless
Include an API gateway in your architecture diagram
Start with a monolith unless the problem explicitly requires microservices
If you choose microservices, explain why (scaling, team autonomy, fault isolation)
Know the difference between a reverse proxy and an API gateway

Key Numbers to Remember:

CODE

Single server capacity:
- 1,000-10,000 concurrent connections
- 500-5,000 requests per second
- 8-64 GB RAM typical

When to split database from app server: ~1,000+ DAU
When to add load balancer: ~10,000+ DAU
When to consider microservices: ~100,000+ DAU or 10+ engineers