AWS Infrastructure Automation with Terraform

Course: AWS Zero to Hero — AWS Simplified
Chapter: Day-24 | Terraform with AWS Project
Instructors: Abhishek Veeramalla & Nasimulla (Cloud Champ)

Overview

Manual infrastructure provisioning through the AWS Management Console is inherently error-prone, slow, and impossible to scale reliably. While CloudFormation provides AWS-native automation, HashiCorp Terraform has established itself as the industry standard for cloud-agnostic Infrastructure as Code (IaC).

This guide details a production-ready AWS deployment built completely declarative through Terraform. You will construct a multi-availability zone network architecture containing a custom Virtual Private Cloud (VPC), public subnets, an Internet Gateway, route tables, dynamic security groups, an S3 bucket, auto-bootstrapped EC2 web servers, and an Application Load Balancer (ALB) distributing real-time HTTP traffic.

1. Declarative IaC vs. Manual AWS Operations

Provisioning infrastructure via the AWS Management Console requires manual point-and-click operations across dozens of UI screens. In contrast, Infrastructure as Code (IaC) defines modern cloud architecture in declarative configuration files using HashiCorp Configuration Language (HCL).

Key Differences in Workflow Execution

2. Authentication, Provider Configuration & Environment Setup

Terraform relies on Providers—plugins that interact with cloud provider APIs. To interact with AWS, Terraform authenticates using AWS Identity and Access Management (IAM) API access keys.

Security Best Practice: Credential Management

Warning: Never hardcode secret access keys directly inside Terraform configuration files (.tf). Hardcoding credentials risks accidental commit to public version control repositories like GitHub.

Instead of hardcoding, configure credential profiles locally using the AWS Command Line Interface (AWS CLI), which stores them securely in ~/.aws/credentials.

# Verify AWS CLI installation
aws --version

# Authenticate local shell with AWS IAM credentials
aws configure

When prompted, enter your Access Key ID, Secret Access Key, Default Region (e.g., us-east-1), and Output Format (json or none).

Provider Block Definition (provider.tf)

Create the provider definition file to specify the required AWS plugin version and target deployment region:

terraform {
  required_version = ">= 1.2.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

3. Designing the Base Network Stack (VPC & Subnets)

To establish an isolated network boundary, define a custom Virtual Private Cloud (VPC) and divide it into subnets across separate Availability Zones (AZs) for fault tolerance.

Mathematical Subnet Calculation

A $/16$ Classless Inter-Domain Routing (CIDR) block provides $2^{32-16} = 65,536$ total IP addresses:

Splitting this into $/24$ subnets yields $2^{32-24} = 256$ IP addresses per subnet:

HCL Code Implementation

Variables Configuration (variables.tf)

variable "vpc_cidr" {
  description = "Base CIDR block for custom VPC"
  type        = string
  default     = "10.0.0.0/16"
}

Network Definition (main.tf)

# Create Custom VPC
resource "aws_vpc" "my_vpc" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "Project-VPC"
  }
}

# Create Public Subnet 1 in us-east-1a
resource "aws_subnet" "sub1" {
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = "10.0.0.0/24"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = true

  tags = {
    Name = "Public-Subnet-1a"
  }
}

# Create Public Subnet 2 in us-east-1b
resource "aws_subnet" "sub2" {
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = "us-east-1b"
  map_public_ip_on_launch = true

  tags = {
    Name = "Public-Subnet-1b"
  }
}

4. Enabling Internet Connectivity (IGW & Routing)

Subnets created inside a custom VPC remain private by default. To convert subnets into public subnets capable of communicating with the internet, attach an Internet Gateway (IGW) and create a Route Table containing an egress default route (0.0.0.0/0).

HCL Infrastructure Setup

# Create Internet Gateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.my_vpc.id

  tags = {
    Name = "Project-IGW"
  }
}

# Create Route Table with Default Route to Internet Gateway
resource "aws_route_table" "my_rt" {
  vpc_id = aws_vpc.my_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "Public-Route-Table"
  }
}

# Associate Route Table with Subnet 1
resource "aws_route_table_association" "rta1" {
  subnet_id      = aws_subnet.sub1.id
  route_table_id = aws_route_table.my_rt.id
}

# Associate Route Table with Subnet 2
resource "aws_route_table_association" "rta2" {
  subnet_id      = aws_subnet.sub2.id
  route_table_id = aws_route_table.my_rt.id
}

5. Security Groups & S3 Storage Isolation

Security Group Definition

Security groups act as virtual firewalls controlling inbound and outbound network traffic at the instance level.

• Inbound Rules (Ingress): Allow HTTP (Port 80) and SSH (Port 22) traffic from anywhere (0.0.0.0/0).
• Outbound Rules (Egress): Allow all unrestricted outbound connections (-1 protocol, Port 0).

resource "aws_security_group" "web_sg" {
  name_prefix = "web-sg-"
  description = "Allow HTTP and SSH ingress traffic"
  vpc_id      = aws_vpc.my_vpc.id

  # HTTP Ingress
  ingress {
    description = "HTTP Traffic"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # SSH Ingress
  ingress {
    description = "SSH Access"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Outbound Rule
  egress {
    description = "Allow all egress traffic"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "Web-Security-Group"
  }
}

Amazon S3 Object Storage

Simple Storage Service (S3) buckets require globally unique names across all AWS accounts worldwide.

resource "aws_s3_bucket" "proj_bucket" {
  bucket        = "abhishek-cloudchamp-tf-demo-2023-unique"
  force_destroy = true

  tags = {
    Name        = "AppStorageBucket"
    Environment = "Dev"
  }
}

6. EC2 Compute Provisioning with User Data Bootstrapping

Deploy two EC2 instances into separate subnets. Instances utilize User Data scripts to automatically update systems, install Apache HTTP servers, query Instance Metadata Service (IMDS) endpoints for instance details, and serve custom web traffic upon initialization.

User Data Shell Scripts

Script 1 (user_data.sh)

#!/bin/bash
sudo apt-get update -y
sudo apt-get install -y apache2
systemctl start apache2
systemctl enable apache2

INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
echo "<h1>Welcome to Abhishek's Channel</h1><p>Instance ID: $INSTANCE_ID</p>" > /var/www/html/index.html

Script 2 (user_data1.sh)

#!/bin/bash
sudo apt-get update -y
sudo apt-get install -y apache2
systemctl start apache2
systemctl enable apache2

INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
echo "<h1>Welcome to Cloud Champ</h1><p>Instance ID: $INSTANCE_ID</p>" > /var/www/html/index.html

EC2 HCL Declarations

# Web Server 1 in Subnet 1a
resource "aws_instance" "web_server_1" {
  ami                    = "ami-0261755bbcb8c4a84" # Ubuntu 20.04 LTS (us-east-1)
  instance_type          = "t2.micro"
  subnet_id              = aws_subnet.sub1.id
  vpc_security_group_ids = [aws_security_group.web_sg.id]
  user_data              = base64encode(file("user_data.sh"))

  tags = {
    Name = "WebServer-1"
  }
}

# Web Server 2 in Subnet 1b
resource "aws_instance" "web_server_2" {
  ami                    = "ami-0261755bbcb8c4a84" # Ubuntu 20.04 LTS (us-east-1)
  instance_type          = "t2.micro"
  subnet_id              = aws_subnet.sub2.id
  vpc_security_group_ids = [aws_security_group.web_sg.id]
  user_data              = base64encode(file("user_data1.sh"))

  tags = {
    Name = "WebServer-2"
  }
}

7. High Availability with Application Load Balancer

An Application Load Balancer (ALB) operates at Layer 7 (Application Layer) of the OSI model. It inspects incoming HTTP/HTTPS traffic and distributes it round-robin across attached EC2 target instances.

Complete Load Balancer Stack HCL

# Application Load Balancer Infrastructure
resource "aws_lb" "my_alb" {
  name               = "my-app-load-balancer"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.web_sg.id]
  subnets            = [aws_subnet.sub1.id, aws_subnet.sub2.id]

  tags = {
    Environment = "Production"
  }
}

# Target Group Definition
resource "aws_lb_target_group" "my_tg" {
  name     = "my-alb-target-group"
  port     = 80
  protocol = "HTTP"
  vpc_id   = aws_vpc.my_vpc.id

  health_check {
    path                = "/"
    port                = "traffic-port"
    protocol            = "HTTP"
    healthy_threshold   = 3
    unhealthy_threshold = 3
    timeout             = 5
    interval            = 30
    matcher             = "200"
  }
}

# Target Attachment: Server 1
resource "aws_lb_target_group_attachment" "attach_server_1" {
  target_group_arn = aws_lb_target_group.my_tg.arn
  target_id        = aws_instance.web_server_1.id
  port             = 80
}

# Target Attachment: Server 2
resource "aws_lb_target_group_attachment" "attach_server_2" {
  target_group_arn = aws_lb_target_group.my_tg.arn
  target_id        = aws_instance.web_server_2.id
  port             = 80
}

# ALB HTTP Listener Rule
resource "aws_lb_listener" "alb_listener" {
  load_balancer_arn = aws_lb.my_alb.arn
  port              = 80
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.my_tg.arn
  }
}

# Output Public Load Balancer Endpoint
output "load_balancer_dns" {
  description = "Public DNS Endpoint of Application Load Balancer"
  value       = aws_lb.my_alb.dns_name
}

8. Complete Terraform Execution Lifecycle

Execution follows a disciplined lifecycle of syntax validation, static planning, provision execution, and clean resource destruction.

Command Lifecycle Breakdown

Terminal Verification Workflow

# Step 1: Initialize working environment
terraform init

# Step 2: Format and validate configuration code
terraform fmt
terraform validate

# Step 3: Run static dry-run plan
terraform plan

# Step 4: Apply infrastructure changes automatically
terraform apply --auto-approve

# Step 5: Test ALB DNS response via cURL loop
curl http://<load_balancer_dns_output>

# Step 6: Destroy infrastructure to prevent AWS billing charges
terraform destroy --auto-approve

9. Troubleshooting & Production Best Practices

Real-World Engineering Debugging Scenarios

1. Duplicate Resource Identifiers (already declared):

   • Root Cause: Identical resource block names assigned across HCL configurations.
   • Resolution: Ensure every HCL resource block possesses a unique local identifier (e.g., aws_subnet.sub1 vs aws_subnet.sub2).

2. Access Denied / Authentication Errors:

   • Root Cause: Missing, invalid, or scope-restricted IAM API access credentials.
   • Resolution: Run aws configure to verify credentials. Ensure your IAM user possesses appropriate permissions (AmazonVPCFullAccess, AmazonEC2FullAccess, AmazonS3FullAccess).

3. Inaccessible Web Service (Connection Timed Out):

   • Root Cause: Security Group rules missing inbound port 80 access, or missing Route Table Gateway association.
   • Resolution: Inspect aws_security_group.web_sg rules for port 80 Ingress, ensure subnets have public IP mapping enabled (map_public_ip_on_launch = true), and verify route table contains 0.0.0.0/0 targeting the Internet Gateway.

Production Readiness Checklist

• Remote State Management: Never store state locally on personal workstations for team environments. Store terraform.tfstate remotely in an encrypted Amazon S3 bucket with DynamoDB State Locking to prevent concurrent execution conflicts.
• Sensitive Files (.gitignore): Add sensitive environment state and variable files to your version control ignore list:
  GITIGNORE

  Copy

  .terraform/
  *.tfstate
  *.tfstate.backup
  *.tfvars
  

• Modular Infrastructure: Break monolithic code blocks (main.tf) into structured Terraform Modules (e.g., Network Module, Compute Module, Database Module) to enable cross-environment code reusability.

Summary and Key Takeaways

• Infrastructure as Code Standard: Terraform provides cloud-agnostic, declarative infrastructure automation managed by human-readable HCL definitions.
• Credential Protection: Never commit IAM secret keys inside .tf scripts; always utilize ambient terminal credentials configured via aws configure.
• State Management Core: Terraform tracks real-world assets against declared configuration using local or remote terraform.tfstate files.
• Subnet High Availability: Deploy compute nodes across independent Availability Zones within custom VPCs to withstand physical data center outages.
• Dynamic Traffic Distribution: Application Load Balancers inspect OSI Layer 7 traffic and perform automated health checks to route requests exclusively to healthy instances.
• Automated Bootstrapping: Combine compute provisioning with system scripts using base64-encoded user data to launch pre-configured web workloads seamlessly.
• Cost Control Hygeine: Practice disciplined cloud environment cleanup by systematically tearing down temporary learning infrastructure using terraform destroy.