Terraform Zero to Hero: Day 5 – Most Common Terraform Task Used in Real-Time (Terraform Provisioners)

This chapter focuses on a real-world DevOps task: automating infrastructure provisioning and application deployment using Terraform Provisioners. Instead of jumping straight into abstract definitions, we examine a real-world developer workflow, implement an end-to-end Terraform project from scratch, and analyze how provisioners bridge the gap between infrastructure creation and software configuration.

1. Context and State Management Recap (Day 4 Review)

Before diving into provisioners, it is essential to recall the core state management principles covered in the previous session:

• State File Fundamentals: Terraform uses a state file (terraform.tfstate) as the "heart and memory" to map real-world cloud resources to your configuration files.
• Why Version Control Fails for State: Storing state files directly in Git repositories creates major security vulnerabilities (secrets exposure) and severe locking conflicts during concurrent team modifications.
• Remote Backends & Locking: Using cloud object storage (such as AWS S3) paired with a distributed locking mechanism (AWS DynamoDB) prevents concurrent race conditions and provides secure team collaboration.

2. Real-World Problem Statement: The Development Team Workflow

The Scenario

Imagine an application development team (Team XYZ) working on a Python Flask application (app.py). Whenever developers modify their application code, they need a dedicated testing environment to verify their changes before merging to production.

The Manual Bottleneck vs. The DevOps Automation Goal

• Manual Approach: A developer or engineer manually logs into the AWS Console, provisions a VPC, configures subnets, sets up route tables, launches an EC2 instance, configures security groups, SCPs the application file, installs Python dependencies, and starts the service. This manual loop takes around 1 hour per developer.
• DevOps Zero-Touch Automation Goal: Write a unified Terraform project that handles infrastructure creation and software deployment in a single command, saving countless hours across multiple projects and developers.

3. Step-by-Step Terraform Infrastructure Blueprint

To write a clean Terraform project without fear, always map out the resources manually first. The configuration must construct the network topology and compute instances in the correct dependency order.

Complete main.tf Architecture

provider "aws" {
  region = "us-east-1"
}

# 1. Create a Custom VPC
resource "aws_vpc" "myvpc" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "dev-vpc"
  }
}

# 2. Generate/Upload an SSH Key Pair
resource "aws_key_pair" "example" {
  key_name   = "terraform-deploy-key"
  public_key = file("~/.ssh/id_rsa.pub")
}

# 3. Create a Public Subnet inside the VPC
resource "aws_subnet" "sub1" {
  vpc_id                  = aws_vpc.myvpc.id
  cidr_block              = "10.0.1.0/24"
  map_public_ip_on_launch = true
  availability_zone       = "us-east-1a"
}

# 4. Create Internet Gateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.myvpc.id
}

# 5. Create Custom Route Table and Route to IGW
resource "aws_route_table" "rt" {
  vpc_id = aws_vpc.myvpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }
}

# 6. Associate Route Table with Subnet
resource "aws_route_table_association" "rta" {
  subnet_id      = aws_subnet.sub1.id
  route_id       = aws_route_table.rt.id
}

# 7. Create Security Group (Allow SSH & HTTP)
resource "aws_security_group" "webSg" {
  name   = "vpc-sg"
  vpc_id = aws_vpc.myvpc.id

  ingress {
    description = "Allow SSH"
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    description = "Allow HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# 8. Create EC2 Instance with Provisioners
resource "aws_instance" "server" {
  ami                    = "ami-0c7217cdde317cfec" # Ubuntu 22.04 LTS (example)
  instance_type          = "t2.micro"
  key_name               = aws_key_pair.example.key_name
  subnet_id              = aws_subnet.sub1.id
  vpc_security_group_ids = [aws_security_group.webSg.id]

  # Connection block for Provisioners
  connection {
    type        = "ssh"
    user        = "ubuntu"
    private_key = file("~/.ssh/id_rsa")
    host        = self.public_ip
  }

  # File Provisioner: Copy app.py to remote instance
  provisioner "file" {
    source      = "app.py"
    destination = "/home/ubuntu/app.py"
  }

  # Remote Exec Provisioner: Install dependencies and run application
  provisioner "remote-exec" {
    inline = [
      "sudo apt update -y",
      "sudo apt install -y python3-pip",
      "cd /home/ubuntu",
      "sudo pip3 install flask",
      "nohup sudo python3 app.py &"
    ]
  }
}

4. Understanding Terraform Provisioners

What Are Provisioners?

By design, Terraform's primary goal is infrastructure provisioning (creating VPCs, subnets, and virtual machines). However, infrastructure without configuration is incomplete. Provisioners allow you to execute scripts, copy files, or bootstrap configuration management tools (such as Ansible or Chef) directly on or against a newly created resource during its lifecycle.

Types of Provisioners in Terraform

5. Lifecycle Phases of Provisioners

By default, provisioners run during the creation of a resource. However, they can be configured to run during destruction as well.

• Creation-Time Provisioners: Executed after the resource is created. If a creation-time provisioner fails, the resource is tainted and marked as destroyed on the next apply.
• Destruction-Time Provisioners: Executed before a resource is destroyed. Enabled by adding when = destroy inside the provisioner block.

6. Execution Workflow & Practical Demonstration

Executing the project requires a disciplined CLI lifecycle to prevent dangling resources or cloud billing charges.

Step-by-Step Terminal Execution

1. Initialize the working directory (downloads required provider plugins):

   BASH

   Copy

   terraform init
   

2. Verify the execution plan (ensures all 8 resources are targeted):

   BASH

   Copy

   terraform plan
   

3. Apply the configuration:

   BASH

   Copy

   terraform apply --auto-approve
   

4. Verify running processes and troubleshoot live: If a background process fails to bind or persist, log in manually via SSH to inspect the instance:

   BASH

   Copy

   ssh -i ~/.ssh/id_rsa ubuntu@<EC2-PUBLIC-IP>
   sudo python3 app.py
   

5. Clean up resources to avoid AWS charges:

   BASH

   Copy

   terraform destroy --auto-approve
   

7. Industry Best Practices & Limitations of Provisioners

While provisioners are convenient for small tasks or demos, HashiCorp recommends treating them as a last resort for production environments.

Why Provisioners Are a "Last Resort" (The HashiCorp Warning)

• Lack of State Awareness: Terraform cannot track software installed inside an EC2 instance via remote-exec. If a python package is uninstalled manually on the server, Terraform remains unaware of the drift.
• Connection Fragility: Provisioners rely heavily on active network connectivity (SSH/WinRM). If security groups misconfigure SSH access or network ACLs block ports, the entire Terraform apply fails midway.
• Idempotency Challenges: Re-running terraform apply with provisioners can lead to duplicate executions or script failures if scripts are not written idempotently.

Preferred Modern Alternatives

• Cloud-Init & User Data: Pass lightweight bootstrapping scripts directly into the AWS EC2 user_data argument for initial configuration.
• Custom AMIs (Golden Images): Bake applications and dependencies into pre-configured machine images using tools like Packer before provisioning infrastructure.
• Configuration Management Tools: Offload post-provisioning configuration to dedicated tools like Ansible, Chef, or Puppet triggered via CI/CD pipelines after Terraform finishes building infrastructure.

Summary and Key Takeaways

• Core Purpose: Terraform provisioners bridge the gap between cloud infrastructure provisioning and software configuration.
• File Provisioner: Enables seamless file transfer (such as application source code app.py) from the local runner to remote target instances.
• Remote-Exec Provisioner: Executes shell commands remotely over SSH to install dependencies, update package managers, and launch services.
• Local-Exec Provisioner: Executes commands locally on the machine executing Terraform, useful for logging, notifications, or triggering local automation scripts.
• Dependency Order: Always design infrastructure from the ground up—VPC $\rightarrow$ Subnet $\rightarrow$ Internet Gateway $\rightarrow$ Route Table $\rightarrow$ Security Group $\rightarrow$ EC2 Instance.
• Real-World Impact: Automating these workflows achieves "zero-touch automation," saving developers hours of manual testing environment setup.
• Production Caution: Use provisioners sparingly; favor Golden AMIs (Packer), User Data, or dedicated configuration management tools like Ansible for enterprise-grade environments.
• Resource Hygiene: Always execute terraform destroy after testing experimental projects to prevent unexpected cloud infrastructure billing.