SLIDE 88: The Latency Problem at Global Scale
Section 8: Content Delivery Networks
In Section 2, we established that the speed of light in fiber optic cable is approximately 200,000 km/s, and that a round trip from Mumbai to San Francisco takes a minimum of ~136ms in theory and 200-400ms in practice. No amount of server optimization can overcome the physics of distance. A CDN solves this by placing copies of your content on servers distributed around the world, so that users fetch data from a location nearby instead of from your origin data center.
What We’ll Cover:
- CDN architecture (origin and edge servers)
- CDN request flow
- Push vs pull CDN
- Cache control and invalidation
- CDN cost structure
- When a CDN is unnecessary
The Problem CDNs Solve:
Scale of CDN Infrastructure:
SLIDE 89: CDN Architecture: Origin and Edge Servers
CDN Architecture
A CDN consists of two types of servers: the origin server (your actual server where the original content lives) and edge servers (CDN servers distributed globally that cache copies of your content).
Terminology:
What CDNs Cache:
SLIDE 90: CDN Request Flow
CDN Request Flow: Step by Step
Path A: Cache Hit (90%+ of requests for popular content)
Edge server has the file, TTL has not expired
Edge server returns the image immediately
Total latency: 10-30ms
Response headers:
HTTP/1.1 200 OK
X-Cache: HIT
Age: 3600 (cached for 1 hour)
Cache-Control: public, max-age=86400
Path B: Cache Miss (first request or expired content)
After the first miss, all subsequent requests from users near Mumbai are served from the edge. A popular thumbnail might be requested millions of times but only fetched from origin once per edge location.
SLIDE 90B: With CDN vs Without CDN: Performance Comparison
Performance Comparison:
Real Impact on Page Load:
YouTube homepage without CDN:
HTML from origin: 200ms
50 static assets from origin: 200ms each (parallel, limited connections)
Total: ~1-3 seconds (multiple round trips to distant origin)
YouTube homepage with CDN:
HTML from origin: 200ms (not cached, personalized)
50 static assets from CDN edge: 20ms each (nearby, fast)
Total: ~300-500ms
The difference: 1-3 seconds vs 300-500ms.
This directly impacts user engagement.
Google's research: 53% of mobile users abandon sites that take
longer than 3 seconds to load.
SLIDE 91: Push CDN vs Pull CDN
Push CDN vs Pull CDN
There are two models for how content gets from your origin server to CDN edge servers.
Pull CDN (most common):
Content is fetched by the CDN on demand when a user requests it.
1. User requests image from CDN edge
2. Edge doesn't have it (cache miss)
3. Edge pulls the image from origin
4. Edge caches it and serves to user
5. Subsequent requests for same image served from edge cache
Characteristics:
- Content only cached after first request
- First request is slow (origin fetch)
- Simple to set up (just point CDN at your origin)
- Unused content is never cached (saves storage)
- Most CDN providers default to pull mode
Push CDN:
Content is uploaded to the CDN by your application proactively.
1. Application creates/updates content
2. Application pushes content to CDN edge servers
3. Content is available on CDN before any user requests it
4. User requests content, edge already has it (always a hit)
Characteristics:
- No cold start (content pre-populated)
- More complex (your app must manage CDN uploads)
- Wastes storage if content is pushed but never requested
- Better for predictable, known content
Comparison:
In Practice:
Pull CDN is the default for nearly all web applications. Push CDN is used for specific cases where you need guaranteed availability on the first request (product launch pages, app binary downloads, firmware updates).
SLIDE 92: CDN Cache Control and Invalidation
CDN Cache Control
Cache behavior is controlled by HTTP headers sent from the origin server. These headers tell the CDN how long to cache content and under what conditions.
Key Cache-Control Headers:
Cache-Control: public, max-age=86400
public: CDN (and browser) can cache this
max-age=86400: Cache for 86,400 seconds (24 hours)
Cache-Control: private, no-cache
private: Only the user's browser can cache, NOT the CDN
no-cache: Must revalidate with origin before serving cached copy
Cache-Control: no-store
Do not cache at all (sensitive data, personalized responses)
Cache-Control Decision Table:
CDN Invalidation (Purging):
SLIDE 93: CDN at Scale: Real-World Numbers
CDN at Instagram Scale
Instagram's media delivery:
- 2 billion monthly active users
- 95 million photos/videos uploaded per day
- Average image size: 2 MB (compressed)
- Average views per image: 100 (varies wildly)
Daily egress without CDN:
10 billion image views/day × 2 MB = 20,000 TB/day
20,000 TB / 86,400 seconds = 231 GB/sec of continuous bandwidth
This is physically impossible from a single data center.
Even 10 data centers would each need to sustain 23 GB/sec.
With CDN (80% cache hit rate):
Origin serves: 20% of 231 GB/sec = 46 GB/sec
CDN edge serves: 80% of 231 GB/sec = 185 GB/sec (distributed globally)
No single location handles more than a fraction of the total load.
CDN Performance Metrics:
SLIDE 94: CDN Cost Structure
CDN Cost
CDN pricing is primarily based on bandwidth (data transferred from edge servers to users). At scale, CDN bandwidth is the single largest infrastructure cost for media-heavy applications.
AWS CloudFront Pricing (approximate):
Cost Example at Instagram Scale:
Monthly egress via CDN: ~600 PB (600,000 TB)
At negotiated enterprise pricing (~$0.02/GB):
600,000 TB × 1,000 GB/TB × $0.02/GB = $12,000,000/month
Compare to serving directly from origin (AWS EC2 egress):
600,000 TB × $0.09/GB = $54,000,000/month
CDN saves: $42M/month ($504M/year)
CDN is not just a performance tool. At scale, it's a cost
reduction tool. Edge delivery is cheaper than origin egress.
Cost Optimization Strategies:
1. Maximize cache hit rate
Higher hit rate = less origin egress = lower total cost
Improve from 80% to 90%: cuts origin traffic in half
2. Compress content
Enable gzip/brotli compression for text-based assets (HTML, CSS, JS)
Typical compression: 60-80% size reduction
Smaller files = less bandwidth = lower cost
3. Serve appropriate image sizes
Mobile device: serve 640px image, not 4K
Responsive images: <img srcset="small.jpg 640w, large.jpg 1080w">
Reduces bandwidth 50-75% for mobile users
4. Use versioned URLs with long TTLs
styles.abc123.css with max-age=1 year
No unnecessary revalidation requests
Reduces origin requests to near zero for static assets
SLIDE 95: When a CDN is Unnecessary
When You Don’t Need a CDN
A CDN is not always the right solution. Adding one introduces complexity (cache invalidation, debugging cache behavior, additional vendor dependency) and cost. For some systems, the benefits don’t justify these costs.
Skip CDN When:
Use CDN When:
Interview Tip:
In a system design interview, include a CDN in your architecture whenever the system serves static content (images, videos) to a global user base. If the interviewer specifies a single-region system or a purely dynamic API, you can note that a CDN is optional for that specific case. Showing that you know when NOT to use a technology is as valuable as knowing when to use it.
SLIDE 96: Section Summary and Interview Tips
Section 8 Summary: Content Delivery Networks
Concepts Covered:
Key Numbers to Remember:
- Edge server latency: 10-30ms (vs 200-400ms from distant origin)
- Target cache hit rate: 80%+ (production: often 90-99%)
- Purge propagation: 30 seconds to 10 minutes depending on provider
- CDN cost: $0.02-0.085/GB depending on volume
- Instagram CDN egress: ~600 PB/month
- CDN savings vs origin: 4-5x cheaper per GB at scale
How This Connects to Other Sections:
Section 2 (Internet): CDN uses DNS (CNAME records) for edge routing
Section 5 (Scaling): CDN is Stage 6 of the scaling progression
Section 6 (Caching): CDN is a specialized cache at the edge layer
Section 7 (Load Balancing): GSLB routes users to nearest CDN edge
Section 13 (Design): URL shortener architecture may include CDN for redirect performance