Terraform with AWS: End-to-End Real-Time Infrastructure Deployment

1. Introduction & Project Architecture Overview

Automating cloud infrastructure using Terraform replaces manual console-clicking with reproducible, version-controlled configuration files (Infrastructure as Code - IaC). In this project, we transition a manual AWS 3-tier architecture—previously built via the AWS Management Console in Day 7 of the course—into a fully automated Terraform configuration.

Architectural Blueprint

Structure & Concept Flow

Internet

Internet Gateway (IGW)

Application Load Balancer (ALB)

+---> Public Subnet 1 (US East 1A) ---> EC2 Instance 1 (Apache / Welcome App 1)

+---> Public Subnet 2 (US East 1B) ---> EC2 Instance 2 (Apache / Welcome App 2)

2. Authentication & Prerequisites Setup

Before Terraform can communicate with AWS APIs, authentication and authorization must be established using an IAM user with appropriate permissions (ideally administrative privileges for learning, transitioning to least-privilege for production).

Step-by-Step Authentication Workflow:

Create an IAM User: Log into the AWS Console, navigate to IAM, create a user with appropriate permissions (e.g., EC2, S3, VPC, ELB access).
Generate Access Keys: Navigate to Security Credentials -> Access Keys -> Select Command Line Interface (CLI).
Configure AWS CLI: Run the configuration utility in your terminal to securely store credentials locally.
BASH
```
aws configure
```
- AWS Access Key ID: YOUR_ACCESS_KEY
- AWS Secret Access Key: YOUR_SECRET_KEY
- Default region name: us-east-1
- Default output format: none

3. Initializing the Terraform Project & Provider Configuration

Terraform relies on Providers to interact with cloud platforms, SaaS providers, and APIs. The provider plugin translates your HCL (HashiCorp Configuration Language) code into raw API calls.

Directory Structure & Initialization

Create a dedicated project directory and open it in your code editor (e.g., VS Code with the official HashiCorp Terraform extension):

BASH

mkdir terraform-aws-project
cd terraform-aws-project
code .

Writing `provider.tf`

Create a configuration file to declare the required provider version and target region:

HCL

terraform {
  required_version = ">= 1.0.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

Initializing Terraform

Run the initialization command to download provider plugins and prepare the working directory:

BASH

terraform init

CODE

Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 4.0"...
- Installing hashicorp/aws v4.x.x...
Terraform has been successfully initialized!

4. Building the Custom Networking Layer (VPC & Subnets)

Rather than using the default VPC, best practices dictate creating a custom Virtual Private Cloud (VPC) with tailored CIDR blocks and subnets across multiple Availability Zones (AZs) for high availability.

Defining Variables (`variables.tf`)

HCL

variable "vpc_cidr" {
  type    = string
  default = "10.0.0.0/16"
}

Declaring Networking Resources (`main.tf`)

HCL

# 1. Custom VPC
resource "aws_vpc" "my_vpc" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "custom-vpc-terraform"
  }
}

# 2. Public Subnet 1 (AZ: us-east-1a)
resource "aws_subnet" "sub_1" {
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = true

  tags = {
    Name = "public-subnet-1"
  }
}

# 3. Public Subnet 2 (AZ: us-east-1b)
resource "aws_subnet" "sub_2" {
  vpc_id                  = aws_vpc.my_vpc.id
  cidr_block              = "10.0.2.0/24"
  availability_zone       = "us-east-1b"
  map_public_ip_on_launch = true

  tags = {
    Name = "public-subnet-2"
  }
}

5. Internet Gateway & Routing Configuration

To allow public subnets to communicate with the external internet, an Internet Gateway (IGW) must be attached to the VPC, and a custom Route Table must route all outbound traffic (0.0.0.0/0) through that IGW.

Adding Gateway & Route Table Resources

HCL

# 1. Internet Gateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.my_vpc.id

  tags = {
    Name = "main-igw"
  }
}

# 2. Custom Route Table
resource "aws_route_table" "my_rt" {
  vpc_id = aws_vpc.my_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "public-route-table"
  }
}

# 3. Route Table Association - Subnet 1
resource "aws_route_table_association" "rta_1" {
  subnet_id      = aws_subnet.sub_1.id
  route_table_id = aws_route_table.my_rt.id
}

# 4. Route Table Association - Subnet 2
resource "aws_route_table_association" "rta_2" {
  subnet_id      = aws_subnet.sub_2.id
  route_table_id = aws_route_table.my_rt.id
}

6. Security Groups & S3 Bucket Integration

Security groups act as virtual firewalls for EC2 instances. We will allow inbound traffic on Port 80 (HTTP) for web traffic and Port 22 (SSH) for remote management.

Security Group Definition

HCL

resource "aws_security_group" "web_sg" {
  name        = "web-sg"
  description = "Allow HTTP and SSH inbound traffic"
  vpc_id      = aws_vpc.my_vpc.id

  # HTTP Inbound Rule
  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # SSH Inbound Rule
  ingress {
    description = "SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Outbound Rule (All traffic allowed)
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "web-security-group"
  }
}

S3 Bucket Resource

HCL

resource "aws_s3_bucket" "example_bucket" {
  bucket        = "abhishek-terraform-project-bucket-2023-unique"
  force_destroy = true

  tags = {
    Environment = "Dev"
  }
}

7. EC2 Instances & Automated Bootstrap Scripting (User Data)

We configure two separate Ubuntu EC2 instances launched in distinct subnets. Each instance runs an automated shell script (user-data.sh) upon startup to install Apache and display a customized welcome message referencing the instance ID and metadata.

Bootstrap Script (`user-data.sh`)

BASH

#!/bin/bash
apt-get update -y
apt-get install -y apache2
systemctl start apache2
systemctl enable apache2

# Fetch Instance ID using AWS IMDSv1/v2 metadata
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

echo "<h1>Terraform Project - Welcome to Abhishek's Channel</h1>" > /var/www/html/index.html
echo "<p><strong>Instance ID:</strong> $INSTANCE_ID</p>" >> /var/www/html/index.html

EC2 Resource Definitions (`main.tf`)

HCL

# Server 1 (Subnet 1)
resource "aws_instance" "web_server_1" {
  ami                    = "ami-053b0d53c279acc90" # Ubuntu 20.04 LTS (Region-specific)
  instance_type          = "t2.micro"
  subnet_id              = aws_subnet.sub_1.id
  vpc_security_group_ids = [aws_security_group.web_sg.id]
  user_data              = base64encode(file("user-data.sh"))

  tags = {
    Name = "web-server-1"
  }
}

# Server 2 (Subnet 2 - with distinct greeting script variant)
resource "aws_instance" "web_server_2" {
  ami                    = "ami-053b0d53c279acc90"
  instance_type          = "t2.micro"
  subnet_id              = aws_subnet.sub_2.id
  vpc_security_group_ids = [aws_security_group.web_sg.id]
  user_data              = base64encode(file("user-data-2.sh"))

  tags = {
    Name = "web-server-2"
  }
}

8. Application Load Balancer (ALB) Setup

An Application Load Balancer distributes incoming HTTP requests across both EC2 instances to ensure high availability and load distribution.

ALB Resources & Target Group Attachment

HCL

# 1. Application Load Balancer
resource "aws_lb" "my_lb" {
  name               = "app-load-balancer"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.web_sg.id]
  subnets            = [aws_subnet.sub_1.id, aws_subnet.sub_2.id]

  tags = {
    Name = "my-alb"
  }
}

# 2. Target Group
resource "aws_lb_target_group" "my_tg" {
  name     = "app-target-group"
  port     = 80
  protocol = "HTTP"
  vpc_id   = aws_vpc.my_vpc.id

  health_check {
    path                = "/"
    protocol            = "HTTP"
    port                = "traffic-port"
    matcher             = "200"
    interval            = 30
    timeout             = 5
    healthy_threshold   = 2
    unhealthy_threshold = 2
  }
}

# 3. Target Group Attachment - Server 1
resource "aws_lb_target_group_attachment" "tga_1" {
  target_group_arn = aws_lb_target_group.my_tg.arn
  target_id        = aws_instance.web_server_1.id
  port             = 80
}

# 4. Target Group Attachment - Server 2
resource "aws_lb_target_group_attachment" "tga_2" {
  target_group_arn = aws_lb_target_group.my_tg.arn
  target_id        = aws_instance.web_server_2.id
  port             = 80
}

# 5. ALB Listener
resource "aws_lb_listener" "front_end" {
  load_balancer_arn = aws_lb.my_lb.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.my_tg.arn
  }
}

Output Configuration (`outputs.tf`)

To easily retrieve the ALB DNS name after deployment without checking the console:

HCL

output "alb_dns_name" {
  description = "The DNS name of the Application Load Balancer"
  value       = aws_lb.my_lb.dns_name
}

9. Terraform Execution Lifecycle & Debugging Workflows

Executing Terraform requires a disciplined sequence of validation, planning, application, and teardown commands.

Terraform Execution Command Reference

Command	Purpose	When to Use
`terraform fmt`	Automatically formats HCL configuration files to standard style.	Before committing or planning.
`terraform validate`	Checks syntax, parameter correctness, and references without AWS calls.	After writing code blocks.
`terraform plan`	Generates a dry-run execution plan showing additions, changes, and destructions.	Before applying changes.
`terraform apply`	Provisions or updates actual cloud resources based on state and code.	To deploy infrastructure.
`terraform destroy`	Tears down all resources managed by the current Terraform state file.	To clean up and avoid cloud charges.

Live Debugging Mindset

Syntax Errors: Always inspect error messages carefully; Terraform explicitly points to the file name and line number where evaluation failed.
State Management: Never edit terraform.tfstate manually. Let Terraform manage state reconciliation automatically.
Resource Dependency: Terraform automatically computes dependency graphs (e.g., ensuring the VPC exists before subnets are created), but explicit dependencies can be managed using depends_on when needed.

10. Summary and Key Takeaways

Infrastructure as Code (IaC): Terraform allows developers and engineers to manage infrastructure declaratively, making environments reproducible, trackable, and version-controlled.
Provider Configuration: Providers act as the translation layer between Terraform and cloud APIs (such as AWS, GCP, or Azure).
Modular Code Organization: Separating configuration into logical files (provider.tf, variables.tf, main.tf, outputs.tf) improves maintainability and readability.
Custom Networking Control: Building custom VPCs, subnets, Internet Gateways, and route tables provides fine-grained control over cloud network topologies.
Automated Bootstrapping: Using user-data scripts with base64encode(file(...)) allows automated software installation and configuration upon instance launch.
Load Balancing Integration: Associating Application Load Balancers (ALBs) and Target Groups with EC2 instances ensures high availability and even traffic distribution.
Dry-Run Planning: terraform plan is an essential safety check that previews infrastructure modifications before execution.
Resource Cleanup: Always execute terraform destroy after testing or completing projects to prevent unexpected cloud provider billing charges.

Day-24 | TERRAFORM WITH AWS PROJECT | @cloudchamp | AWS REAL TIME PROJECT #aws

Terraform with AWS: End-to-End Real-Time Infrastructure Deployment

1. Introduction & Project Architecture Overview

Architectural Blueprint

2. Authentication & Prerequisites Setup

Step-by-Step Authentication Workflow:

3. Initializing the Terraform Project & Provider Configuration

Directory Structure & Initialization

Writing provider.tf

Initializing Terraform

4. Building the Custom Networking Layer (VPC & Subnets)

Defining Variables (variables.tf)

Declaring Networking Resources (main.tf)

5. Internet Gateway & Routing Configuration

Adding Gateway & Route Table Resources

6. Security Groups & S3 Bucket Integration

Security Group Definition

S3 Bucket Resource

7. EC2 Instances & Automated Bootstrap Scripting (User Data)

Bootstrap Script (user-data.sh)

EC2 Resource Definitions (main.tf)

8. Application Load Balancer (ALB) Setup

ALB Resources & Target Group Attachment

Output Configuration (outputs.tf)

9. Terraform Execution Lifecycle & Debugging Workflows

Terraform Execution Command Reference

Live Debugging Mindset

10. Summary and Key Takeaways

Writing `provider.tf`

Defining Variables (`variables.tf`)

Declaring Networking Resources (`main.tf`)

Bootstrap Script (`user-data.sh`)

EC2 Resource Definitions (`main.tf`)

Output Configuration (`outputs.tf`)