Day-7 | Load Balancing, Autoscaling, and High Availability in GCP

This lesson covers the core architecture required to build resilient, auto-recovering, and scalable infrastructure on Google Cloud Platform (GCP). Through real-world e-commerce and media streaming scenarios, we examine three interdependent foundational concepts: Load Balancing, High Availability (HA), and Autoscaling. We then walk through a end-to-end practical demonstration using GCP Managed Instance Groups (MIGs), HTTP Application Load Balancers, custom startup scripts, and VPC firewall rules.

1. The Single-Instance Bottleneck (Status Quo)

When launching a web application on Google Cloud, engineers typically start by deploying a compute instance within a single Virtual Private Cloud (VPC) subnetwork and mapping its public IP address to a Domain Name System (DNS) record.

Real-Life Analogy: Walmart’s Early E-Commerce Platform

Imagine Walmart launching its initial e-commerce web platform on GCP. Based on early user demographics, management identifies that the majority of traffic originates from the US East region (us-east1).

To support this workload, the cloud engineer performs the following baseline setup:

1. Provisions a custom VPC in GCP.
2. Enables the Compute Engine API and provisions a single Virtual Machine (VM) instance in us-east1.
3. Assigns virtual hardware resources: $4\text{ VCPUs}$, $4\text{ GB RAM}$, and $100\text{ GB Storage}$.
4. Deploys the Walmart web application code and maps the VM's public IP address to walmart.com via DNS.

The Breaking Point: Traffic Spikes

During standard business operations, this single VM easily handles baseline traffic of $1,000\text{ requests/day}$. However, management announces a major flash sale expected to double or triple traffic instantly to $2,000 - 3,000\text{ requests/day}$.

Because a single virtual machine has finite compute capacity ($4\text{ vCPUs}$), it hits compute saturation. Requests queue up, latency spikes exponentially, and the application eventually crashes.

2. Core Mental Model: What is Load Balancing?

To handle increased load, the cloud engineer provisions a second virtual machine (VM2) with an identical copy of the application. However, exposing two raw IP addresses directly to end users creates a distributed entry-point problem:

• How do we prevent 100% of users from clicking the link to VM1 while VM2 sits idle?
• How can traffic be divided equally ($1,000\text{ requests}$ to VM1 and $1,000\text{ requests}$ to VM2) without exposing underlying infrastructure details to the client?

Definition

A Load Balancer is a managed entry point that sits in front of backend virtual machines and distributes incoming network traffic across multiple running instances of an application based on a configured load-balancing algorithm.

In GCP terminology, the collection of backend compute resources grouped behind a load balancer is managed via a Backend Service (analogous to Target Groups in AWS).

3. Load Balancing Algorithms & Routing Strategies

Modern cloud load balancers do not rely solely on simple round-robin routing; they support sophisticated traffic management algorithms tailored to specific compute capacities and regional layouts.

Common Routing Strategies

1. Round-Robin (Default): Requests are distributed sequentially across all healthy backend instances ($1:1$ distribution).
2. Weighted / Ratio-Based Routing: If VM1 has higher hardware specs ($6\text{ vCPU}, 6\text{ GB RAM}$) than VM2 ($4\text{ vCPU}, 4\text{ GB RAM}$), traffic can be routed according to a proportional ratio such as $6:4$ or $11:10$. The load balancer forwards $\sim 1,200\text{ requests}$ to VM1 and $\sim 800\text{ requests}$ to VM2.
3. Geographic (Geo-Proximity) Routing: Traffic is directed to the data center physically closest to the requesting user to minimize network latency.
4. Least Connections / Capacity Utilization: Traffic is dynamically routed to whichever backend instance currently handles the fewest active connections or lowest CPU utilization.

4. High Availability (HA) & Regional Fault Tolerance

While deploying multiple VMs behind a load balancer solves capacity bottlenecks, it does not automatically guarantee fault tolerance if all VMs reside within the same data center.

Availability Zones vs. Data Center Failure

An Availability Zone (AZ) is an isolated physical data center within a GCP region equipped with independent power, cooling, and networking infrastructure.

If an engineer provisions two VMs inside the exact same zone (e.g., us-east1-a), a local data center outage (caused by power failure, fiber cuts, or natural disasters) will take down both instances simultaneously, rendering the entire application unavailable.

High Availability Architecture

High Availability (HA) is achieved by distributing duplicate application instances across multiple Availability Zones within a region, or across multiple regions globally.

• Regional High Availability: The load balancer routes traffic across VMs located in us-central1-a, us-central1-b, and us-central1-f. If us-central1-a goes down, traffic automatically shifts to the remaining healthy zones.
• Global Load Balancing: Extends HA across continents. One set of backend instances runs in us-east1 (United States), while another runs in asia-south1 (India), with a single Global Application Load Balancer managing multi-region failover and latency-based routing.

5. Autoscaling with Managed Instance Groups (MIG)

Manual instance management fails when traffic demand is unpredictable.

Real-Life Analogy: Unpredictable Streaming Peaks

Consider platforms like HBO during a Game of Thrones premiere, or Disney+ Hotstar / JioCinema when cricketer Virat Kohli walks out to bat during a live match.

Engineers cannot manually predict whether concurrency will spike to $10,000$ or $1,000,000$ concurrent connections, nor can human operators manually spin up infrastructure fast enough during rapid surges.

Managed Instance Groups (MIGs) and Scaling Policy

In GCP, dynamic resource elasticity is managed via a Managed Instance Group (MIG). Instead of creating isolated VM instances manually, engineers configure two core objects:

1. Instance Template: A declarative blueprint specifying the VM specifications (OS distribution, CPU/RAM size, spot/standard provisioning model, disk parameters, network tags, and startup scripts).
2. Managed Instance Group (MIG): A controller that reads the Instance Template, deploys $N$ copies across specified zones, monitors cluster health, and automatically scales the number of instances up or down based on load metrics.

Mathematical Scaling Threshold Rule

An autoscaler operates on policy thresholds defined by parameters like:

• Scale-Out Trigger: When $\text{CPU}_{\text{average}} \ge \theta$, the MIG controller calls the Compute Engine API to spin up a new VM instance using the template until $N = N_{\max}$.
• Scale-In Trigger: When $\text{CPU}_{\text{average}} < \theta - \Delta$, the MIG controller safely terminates idle VMs until $N = N_{\min}$.

6. Architectural Request Flow & Health Check Lifecycle

To ensure traffic is only routed to functional VMs, the load balancer continuously performs automated Health Checks.

The Health Check Mechanism

1. Periodic Probing: Every $5\text{ seconds}$ (default), the load balancer sends an HTTP request (e.g., GET /) to each VM on port 80.
2. Healthy Status ($200\text{ OK}$): The instance remains in the routing pool.
3. Unhealthy Status (Timeout / $5xx\text{ Error}$): If an instance fails consecutive checks, the load balancer marks it unhealthy and instantly stops forwarding client traffic to it.
4. Auto-Healing: The MIG controller automatically terminates the unhealthy instance and provisions a fresh VM from the Instance Template to maintain $N_{\min}$.

7. Hands-On Demo: Step-by-Step GCP UI Implementation

Below is the step-by-step implementation guide to configure an Auto-Scaling, Highly Available, Load-Balanced web application using the GCP Management Console.

Step 1: Define the Compute Engine Instance Template

1. Navigate to Compute Engine $\rightarrow$ Instance Templates $\rightarrow$ Create Instance Template.
2. Set Name: ha-template.
3. Set Location: Regional (Target: us-central1).
4. Set Machine Configuration:
   • Machine Family: General Purpose
   • Series: E2
   • Machine Type: e2-micro ($2\text{ vCPU}, 1\text{ GB RAM}$)
   • Provisioning Model: Spot (minimizes lab costs).
5. Set Firewall: Check Allow HTTP traffic (applies the http-server network tag).
6. Under Advanced Options $\rightarrow$ Management $\rightarrow$ Metadata, paste the following Startup Script:

#!/bin/bash
# Startup script executed as root upon instance boot
apt-get update -y
apt-get install -y nginx

# Write custom HTML page displaying the dynamic instance hostname
HOSTNAME=$(hostname)
cat <<EOF > /var/www/html/index.html
<!DOCTYPE html>
<html>
<head>
    <title>MIG High Availability Demo</title>
</head>
<body>
    <h1>Welcome to MIG Demo</h1>
    <p>Served by Backend Instance: <strong>$HOSTNAME</strong></p>
</body>
</html>
EOF

systemctl restart nginx

7. Click Create.

Step 2: Deploy the Managed Instance Group (MIG)

1. Navigate to Compute Engine $\rightarrow$ Instance Groups $\rightarrow$ Create Instance Group.
2. Select New managed instance group (stateless).
3. Configure settings:
   • Name: ha-mig-demo
   • Instance template: ha-template
   • Location: Select Multiple zones (Region: us-central1). Select zones us-central1-b, us-central1-c, and us-central1-f.
4. Configure Autoscaling Policy:
   • Minimum number of instances: 3
   • Maximum number of instances: 10
   • Autoscaling signal: CPU utilization
   • Target CPU utilization: 60%
5. Click Create.

Verification: Navigate to VM Instances. You will observe Compute Engine automatically creating 3 virtual machines distributed across us-central1-b, us-central1-c, and us-central1-f.

Step 3: Configure the Application Load Balancer

1. Navigate to Network Services $\rightarrow$ Load Balancing $\rightarrow$ Create Load Balancer.
2. Under Application Load Balancer (HTTP/S), click Start configuration.
3. Select Public Facing (External) and Regional Application Load Balancer (Region: us-central1).
4. Backend Configuration:
   • Select Backend Services $\rightarrow$ Create a Backend Service.
   • Name: ha-backend-service.
   • Backend type: Instance group.
   • Select Instance Group: ha-mig-demo.
   • Port numbers: 80.
   • Health Check: Click Create a health check.
     • Name: http-basic-check
     • Protocol: HTTP, Port: 80
     • Click Save and Continue.
5. Frontend Configuration:
   • Name: http-frontend-port-80
   • Protocol: HTTP, Port: 80
6. Review configuration and click Create.

Step 4: Configure the VPC Ingress Firewall Rule

When configuring an application load balancer, backend instances must explicitly allow inbound traffic from the external load balancer network on target port 80.

The Port 80 Firewall Issue

If firewall rules are omitted, SSH testing inside a single VM via curl localhost:80 will succeed locally, but external health checks and browser requests via the Load Balancer IP will fail with 502 Bad Gateway or timeouts.

1. Navigate to VPC Network $\rightarrow$ Firewall $\rightarrow$ Create Firewall Rule.
2. Configure parameters:
   • Name: allow-http-ingress
   • Network: default
   • Direction of traffic: Ingress
   • Action on match: Allow
   • Target tags: http-server
   • Source IPv4 ranges: 0.0.0.0/0
   • Protocols and ports: Specified protocols and ports $\rightarrow$ Check tcp $\rightarrow$ Enter 80.
3. Click Create.

Alternatively, execute this gcloud command in Cloud Shell:

gcloud compute firewall-rules create allow-http-ingress \
    --network=default \
    --action=ALLOW \
    --direction=INGRESS \
    --rules=tcp:80 \
    --source-ranges=0.0.0.0/0 \
    --target-tags=http-server

8. Real-World Production Edge Cases & Debugging

During deployment, cloud engineers commonly run into these troubleshooting scenarios:

Edge Case 1: Health Check Failures ($502$ Bad Gateway)

• Symptom: The Load Balancer IP returns 502 Server Error, and backend instances display an Unhealthy status status in the GCP Console.
• Root Cause: The firewall blocks GCP probe IP ranges (130.211.0.0/22 and 35.191.0.0/16), or the web server service (e.g., Nginx) failed to start inside the VM startup script.
• Debugging Command: Connect to a backend VM via Cloud SSH and check Nginx service status and local responsiveness:
  BASH

  Copy

  sudo systemctl status nginx
  curl -I http://localhost:80
  

Edge Case 2: Load Balancer Sticky Sessions / Browser Caching

• Symptom: Refreshing the browser window repeatedly shows the exact same backend instance name, appearing as if load balancing is broken.
• Root Cause: Web browsers cache HTTP responses and reuse persistent TCP socket connections (Keep-Alive).
• Workaround: Test routing distribution using curl directly in terminal loops to bypass local browser caches:

# Execute a 10-iteration loop hitting the Load Balancer IP
LB_IP="34.x.x.x"  # Replace with actual Load Balancer Frontend IP
for i in {1..10}; do 
  curl -s http://$LB_IP | grep "Served by Backend Instance"; 
  sleep 0.5; 
done

Sample Execution Output:

<p>Served by Backend Instance: <strong>ha-mig-demo-1a2b</strong></p>
<p>Served by Backend Instance: <strong>ha-mig-demo-9x8y</strong></p>
<p>Served by Backend Instance: <strong>ha-mig-demo-3c4d</strong></p>
<p>Served by Backend Instance: <strong>ha-mig-demo-1a2b</strong></p>

9. Conceptual Comparison: Instance Template vs. MIG

To consolidate understanding, keep in mind how GCP separates compute definition from compute management:

Summary and Key Takeaways

• Load Balancers sit in front of application instances to evenly distribute incoming user requests across backend computing resources.
• High Availability (HA) protects against data center failures by deploying application duplicates across multiple Availability Zones within a region or globally.
• Managed Instance Groups (MIGs) handle automated horizontal scaling (Scale-Out/Scale-In) based on defined metric rules (e.g., CPU utilization $\ge 60\%$).
• Instance Templates serve as immutable deployment blueprints containing OS boot details, machine hardware sizing, and root startup scripts.
• Health Checks probe backend instances every few seconds; if an instance fails probes, the load balancer routes traffic away from it, and the MIG replaces it automatically.
• VPC Ingress Firewall Rules must explicitly permit incoming TCP traffic on target application ports (e.g., Port 80) from external source ranges (0.0.0.0/0).
• Terminal verification using curl loops is recommended when testing load balancers to avoid browser connection reuse and local response caching.