Computer Networking & System Design For DevOps in OneShot

Chapter: Elastic Load Balancing (ELB) and High Availability Patterns

System Design is the discipline of architecting software applications so they can scale reliably under heavy load, maintain high availability, and isolate failures. A modern web application is rarely a single monolithic script; it is a distributed web of specialized services, caching layers, load balancers, and persistent data stores designed to handle millions of requests gracefully.

1. What is System Design? (The Core Mental Model)

To transition from a developer who writes code to a system architect who designs infrastructure, you must reframe your definition of an application.

Core Definition:
"System Design is the cost of not thinking before you build an application."

When you build a basic web page without planning for infrastructure, you create a fragile component that breaks the moment real-world traffic hits it. System design represents all the upfront architectural decisions—load balancing, database selection, protocol choices, caching, and compliance—required to ensure an application can grow seamlessly.

2. Deconstructing a Production System: The YouTube Architecture

When an unauthenticated user navigates to youtube.com, the platform appears to be a single video player. Beneath the surface, however, dozens of independent microservices execute concurrently to serve that single request.

Key Service Deconstruction

1. Authentication Service: Handles user credentials, issues secure tokens (e.g., JWT/OAuth), and manages authorization policies before exposing content.
2. Video Upload & Transcoding Service: Ingests raw video formats, breaks them into adaptive bitrate chunks, and distributes them across storage clusters.
3. Notification Service: Listens for event triggers (e.g., channel uploads) and pushes real-time messages to millions of subscribed client devices.
4. Data Persistence Layer: Separates relational metadata (user profiles, subscriptions) from massive analytics/telemetry data stores.

3. Front-End Clones vs. Scalable Production Systems

A common misconception among beginner developers is confusing a front-end UI clone with a full-fledged system.

Real-Life Analogy: The "React Netflix Clone" Fallacy

Creating a single-page React app that fetches static video thumbnails from an open API is a UI prototype, not a System Design implementation.

In contrast, a Production Netflix System handles heterogenous devices (smart TVs, mobile phones, laptops) routing millions of concurrent requests globally:

Architectural Comparison: Prototype vs. Production System

4. The "Localhost" Pitfall vs. Production Realities

Real-Life Analogy: The "Book Keeda" Online Bookstore (2016)

In 2016, a developer set out to create a peer-to-peer book-sharing marketplace named Book Keeda. They built static HTML pages, uploaded photos of books, ran the server on localhost:8080, and declared the application complete.

The Flaw: Running an application on localhost keeps it completely isolated inside your machine's private network stack (127.0.0.1). It is disconnected from the public internet. It lacks routing, payment integration, address validation, security, and persistence.

The Lesson: An application is not a production product until it accounts for network routing, client-side data collection, payment verification, and fault isolation.

To transition Book Keeda into a production system, it requires public network routing, persistent database engines, payment gateway integrations, and edge delivery networks.

5. Requirement-Driven System Architecture

You cannot design a generic, "one-size-fits-all" system. Infrastructure design must strictly follow functional and non-functional requirements.

Case Study: Architecture for an E-Learning Platform (trainwithshubham.com)

Consider an e-learning application offering three distinct content types: Live Streams, Recorded Video Courses, and Static PDF Study Notes. Each workload demands a tailored networking protocol and storage strategy.

1. Live Interactive Streaming

• Protocol / Architecture: Low-latency UDP-based streaming protocols (e.g., WebRTC / RTMP) or dedicated third-party API integrations (e.g., Zoom SDKs).
• Rationale: Real-time interaction prioritizes delivery speed over packet loss recovery. TCP's retransmission overhead introduces unacceptable lag for live sessions.

2. Recorded Video On-Demand (VOD)

• Protocol / Architecture: Chunked HTTP Live Streaming (HLS) backed by Cloud Object Storage (e.g., AWS S3 / Google Cloud Storage).
• Rationale: Pre-recorded videos require reliable, high-bandwidth TCP delivery with progressive loading capabilities.

3. Static Study Material (PDFs & Documents)

• Protocol / Architecture: Content Delivery Network (CDN) edge distribution.
• Rationale: Static files should be cached geographically close to the user to reduce load on the primary origin server and decrease latency.

4. Monetization & Compliance Engine

• Protocol / Architecture: Payment Gateway APIs coupled with region-specific compliance middleware (e.g., automated 18% GST calculation and tax processing).
• Rationale: Financial transactions require strict ACID compliance, secure HTTPS endpoints, and audit trails.

6. Traffic Distribution & High Availability via Load Balancing

When an application scales beyond a single server instance, an Elastic Load Balancer (ELB) acts as the traffic conductor, sitting between incoming client requests and backend server fleets.

Primary Duties of a Load Balancer

1. Traffic Distribution: Spreads incoming network requests evenly across healthy compute nodes using algorithms like Round Robin, Least Connections, or IP Hash.
2. Health Monitoring: Periodically probes backend server endpoints (e.g., /healthz). If an instance fails, the load balancer automatically redirects traffic to healthy nodes.
3. TLS/SSL Termination: Offloads CPU-intensive cryptographic decryption from application servers to the load balancer edge.

7. Operational Workload Synthesis

To design scalable systems, map business requirements directly to technical components and protocols:

8. Complete System Architecture Walkthrough

Below is a complete vertical workflow demonstrating how a client request moves through a fully architected, production-grade platform:

9. Code Example: NGINX Reverse Proxy & Load Balancer Setup

To demonstrate how load balancers operate under the hood, here is a functional configuration for an NGINX reverse proxy acting as a Layer 7 load balancer with active health routing:

# /etc/nginx/nginx.conf

events {
    worker_connections 1024;
}

http {
    # Define the backend pool of application servers
    upstream backend_servers {
        least_conn; # Route requests to the server with the fewest active connections
        
        server app_instance_1:8080 max_fails=3 fail_timeout=30s;
        server app_instance_2:8080 max_fails=3 fail_timeout=30s;
        server app_instance_3:8080 max_fails=3 fail_timeout=30s;
    }

    server {
        listen 80;
        server_name api.trainwithshubham.com;

        # Redirect all HTTP traffic to HTTPS
        return 301 https://$host$request_uri;
    }

    server {
        listen 443 ssl;
        server_name api.trainwithshubham.com;

        ssl_certificate /etc/letsencrypt/live/trainwithshubham/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/trainwithshubham/privkey.pem;

        # Route API requests across the backend application pool
        location / {
            proxy_pass http://backend_servers;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }

        # Route static asset requests directly to a dedicated local directory or CDN edge
        location /static/ {
            alias /var/www/static/;
            expires 30d;
            add_header Cache-Control "public, no-transform";
        }
    }
}

Summary and Key Takeaways

• System Design Philosophy: System design is the foundational planning required to ensure applications survive growth, traffic surges, and infrastructure failures.
• UI vs. Backend Architecture: Designing a client-side user interface (like a React clone) is fundamentally different from building a distributed backend that handles scale, persistence, and state.
• Service Decomposition: Complex platforms like YouTube or Netflix depend on decoupled, specialized microservices (Auth, Video Delivery, Notifications, Analytics) rather than a single monolithic app.
• The "Localhost" Trap: Applications running on localhost ignore critical production realities such as public internet routing, access control, persistent data stores, and third-party integrations.
• Requirement-Driven Protocols: Different workloads require different protocols—Live streams rely on low-latency UDP/WebRTC, recorded video uses HLS over TCP, and static assets require CDN edge caching.
• Elastic Load Balancing: ELBs sit at the edge of backend pools to distribute traffic, offload TLS decryption, and bypass unhealthy application instances via automated health checks.
• Production Overheads: Real-world applications must account for payment processing, tax compliance (e.g., GST calculations), regional legal constraints, and deep system telemetry.