Capacity Planning and Estimation: How much data does YouTube store daily?

Back-of-the-envelope calculations are a fundamental pillar of system design interviews. They allow engineers to logically state the parameters influencing a system's scale, justify infrastructure choices, and explicitly define assumptions. This chapter explores how to estimate core storage, supplementary data, caching requirements, and video processing bandwidth for a hyperscale platform like YouTube.

1. Why Do We Need Back-of-the-Envelope Calculations?

System design questions are intentionally open-ended. Interviewers do not expect exact numbers; rather, they evaluate your structured thinking, estimation heuristics, and ability to handle scale.

Structuring Ambiguity: Breaking down a massive system (like YouTube) into manageable components.
Exposing Bottlenecks: Highlighting whether a system will fail due to storage, network bandwidth, or memory limits.
Defending Assumptions: Allowing you to state parameters clearly so the interviewer can correct or adjust them (e.g., changing daily active users from 1 billion to 500 million).

Common everyday estimates used in interviews:

Estimating the hardware requirements for a global video platform.
Estimating the number of petrol pumps in a major metropolis like Mumbai.

2. Core Mental Model: The Scale of YouTube

To perform calculations for a massive platform, we must anchor our estimations on realistic user metrics and content creation rates.

CODE

1 Billion Active Users (DAU)
       |
       | Creator Ratio (1 creator per 1,000 users)
       v
1 Million New Videos Uploaded Daily
       |
       | Average Video Size (~1 GB)
       v
1 Petabyte (PB) of Raw Video Storage Added Daily

Key Initial Assumptions

Active User Base: 1,000,000,000 (1 billion) daily active users.
Creation Frequency: 1 out of every 1,000 users uploads a video daily.
Daily Upload Volume: $\frac{1,000,000,000}{1,000} = 1,000,000$ (1 million) new videos per day.

3. Storage Requirements: Naive vs. Realistic Video Sizing

Estimating video file size is notoriously tricky because raw video data is massive, whereas compressed video files are optimized for delivery.

The Naive Frame-by-Frame Approach

If we try to calculate video size from first principles using raw frames:

Duration: 10 minutes = 600 seconds.
Frame Rate: ~25 frames per second.
Total Frames: $600 \text{ sec} \times 25 \text{ frames/sec} = 15,000$ frames.
Raw Frame Size: Assume each uncompressed frame is 1 MB.
Total Raw Size: $15,000 \times 1 \text{ MB} = 15,000 \text{ MB} \approx 150 \text{ GB}$ per video.

Note: This naive calculation yields an absurdly large number (150 GB for a 10-minute video) because it ignores modern video compression codecs (like H.264, HEVC, or AV1). In a real interview, this estimate is dangerously inaccurate and requires immediate revision or correction from the interviewer.

The Realistic Production Estimate

Average 10-Minute Video Size: ~1 GB (compressed with standard codecs).
Daily Raw Storage: $1,000,000 \text{ videos/day} \times 1 \text{ GB/video} = 1,000,000 \text{ GB} = 1 \text{ Petabyte (PB)}$ per day.

4. Supplementary Storage Requirements: Redundancy and Metadata

Storing 1 PB of raw video daily is only the tip of the iceberg. Production systems require fault tolerance, backups, and rich relational metadata.

Structure & Concept Flow

Raw Video Storage (1 PB / day)

+---> Primary Storage Cluster

+---> Replica / Backup Cluster (Fault Tolerance)

+---> Multi-Region Replication (CDN Edge / Regional Hubs)

Accounting for Redundancy

Fault Tolerance (RAID / Replication): To prevent data loss from hardware failures, companies typically keep multiple copies of each file (e.g., 3x replication).
Adjusted Daily Storage:

\text{1 PB (Raw)} \times 3 \text{ (Replicas)} = 3 \text{ Petabytes of physical storage added daily}.

Metadata Storage Estimation

Every video requires relational data: title, description, tags, user ID, upload timestamps, comments, likes, and view counts.

Metadata Size per Video: ~2 KB to 10 KB.
Daily Metadata Volume: $1,000,000 \text{ videos} \times 10 \text{ KB} = 10,000,000 \text{ KB} \approx 10 \text{ GB/day}$ .
Conclusion: Metadata storage is negligible compared to video blob storage, but indexing and search (Elasticsearch / database scaling) present separate compute challenges.

5. YouTube Caching Estimation

Not all videos are accessed equally. A tiny fraction of videos go viral and account for the vast majority of views, while most videos are rarely watched. This follows a power-law (Pareto / Long Tail) distribution.

The 80/20 Rule in Video Delivery

Hot Content: 20% of videos generate 80% of total daily traffic.
Caching Strategy: We cache hot videos in memory or high-speed SSD-backed caches (Redis / Memcached / CDN Edge Nodes) to reduce disk I/O and latency.

Bandwidth and Read Throughput Calculation

Assume each active user watches an average of 5 videos per day.
Total Daily Views: $1,000,000,000 \text{ users} \times 5 \text{ views/day} = 5,000,000,000 \text{ (5 billion views/day)}$ .
Average Bitrate for Video Streaming: Assume 2 Mbps (Megabits per second) for standard definition/HD streaming.
Average Video Length Streamed: Users rarely watch a full 10-minute video; assume an average viewing session of 5 minutes (300 seconds).

\text{Data Transferred per View} = 2 \text{ Mbps} \times 300 \text{ seconds} = 600 \text{ Megabits} = 75 \text{ Megabytes (MB)}

\text{Total Daily Network Outbound Bandwidth} = 5,000,000,000 \text{ views} \times 75 \text{ MB} = 375,000,000,000 \text{ MB} \approx 375 \text{ Petabytes/day}.

6. Video Processing and Transcoding Estimation

When a creator uploads a 1 GB video, YouTube cannot serve that single file to every device. Users connect via different devices (smartphones, 4K TVs, budget laptops) and variable network conditions (3G, 4G, 5G, erratic Wi-Fi).

Structure & Concept Flow

Raw Upload (1 GB)

Transcoding Pipeline (Worker Nodes)

+---> 4K Resolution (Ultra HD)

+---> 1080p Resolution (Full HD)

+---> 720p Resolution (HD)

+---> 480p / 360p / 240p (Standard / Mobile)

Transcoding Computational Load

Each uploaded video must be transcoded into multiple resolutions and bitrates (Adaptive Bitrate Streaming - ABR).
Transcoding a 10-minute video into 5-6 different profiles requires significant CPU/GPU compute time.
If transcoding takes 3x real-time on standard hardware:
- 10-minute video $\times 3 = 30$ minutes of compute time per video.
- For $1,000,000$ videos uploaded daily:

\text{Total Compute Required} = 1,000,000 \times 30 \text{ minutes} = 30,000,000 \text{ minutes of CPU time/day} \approx 500,000 \text{ CPU hours per day}.

7. Comparative Synthesis: YouTube System Metrics Summary

System Parameter	Estimated Metric	Operational Rationale / Impact
Daily Active Users (DAU)	1 Billion	Baseline for all read/write traffic calculations.
New Videos Uploaded	1 Million / day	Derived from 0.1% upload ratio among DAU.
Raw Video Storage	1 Petabyte / day	Assuming an average compressed video size of 1 GB.
Replicated Storage	3 Petabytes / day	3x replication factor for fault tolerance and high availability.
Metadata Storage	~10 GB / day	Small footprint compared to blobs, but heavy query indexing load.
Total Daily Video Views	5 Billion	Assuming 5 video watches per user daily.
Network Outbound Bandwidth	~375 Petabytes / day	Driven by streaming bitrates and average watch duration.
Transcoding Compute	~500,000 CPU hours / day	Required to generate multi-resolution adaptive streaming profiles.

Summary and Key Takeaways

Back-of-the-envelope calculations provide a structured framework to justify system design assumptions and handle interviewer constraints.
Anchor on user scale: Starting with Daily Active Users (DAU) allows you to derive creation rates, storage growth, and network bandwidth systematically.
Distinguish raw size from compressed reality: Naive pixel-math (frame-by-frame) fails for media systems; always rely on realistic compressed file size benchmarks (e.g., 1 GB per 10-minute video).
Factor in replication: Raw storage is multiplied by fault-tolerance strategies (such as 3x replication across data centers).
Metadata is lightweight: Relational and document metadata (titles, descriptions) require minimal storage (~10 GB/day) relative to video blobs, but demand high-performance indexing.
Caching dominates read architecture: Applying the 80/20 Pareto principle helps design efficient CDN caching layers for hot content.
Transcoding is CPU-intensive: Generating adaptive bitrate streams (4K down to 240p) requires massive distributed worker pools and substantial compute hours.
Network bandwidth eclipses storage: Outbound streaming bandwidth (hundreds of petabytes daily) far exceeds inbound storage ingestion, dictating global CDN requirements.