Day-3: Terraform Modules — Write Your First Module

1. Why Do We Need Terraform Modules? (The Problem Statement)

In software engineering and infrastructure automation, writing an entire project within a single monolithic file (main.tf) creates massive scaling challenges.

To understand why modules are essential, consider a typical enterprise scenario where a single DevOps team supports development projects by provisioning compute instances, virtual private clouds (VPCs), load balancers, Kubernetes (eks) clusters, and storage buckets. If all infrastructure automation scripts are written inside one massive main.tf file, you face the exact same anti-patterns as monolithic software development:

• Lack of Code Readability & Ownership: New team members or developers taking over a bug fix will spend days reading thousands of lines of unsegmented configuration code. It is impossible to assign clear ownership for specific components.
• Difficult Testing & Maintenance: Making a minor adjustment (e.g., modifying an EC2 variable) risks breaking completely unrelated resources in the same file. Testing requires validating and deploying the entire monolithic block.
• Massive Code Duplication: If multiple development teams (e.g., Team A and Team B) require infrastructure setups, copying and pasting large configuration blocks leads to severe technical debt.

2. Real-Life Analogy: Monolith Software vs. Microservices vs. Terraform Modules

To grasp the concept of Terraform modules, compare software architecture evolution with infrastructure management:

• Monolithic App / Monolithic main.tf: Everything is crammed into a single file. Any change requires recompiling/reapplying the entire system.
• Microservices / Terraform Modules: Encapsulated, reusable, version-controlled components that solve specific infrastructure needs and can be plugged into any project safely.

3. Advantages of Using Terraform Modules

When you transition your Terraform projects to a modular structure, you unlock several core enterprise benefits:

• Modularity & Encapsulation: Group related resources together so that each folder/module manages a specific domain (e.g., compute, networking).
• Reusability: Write once, use everywhere. A tested and verified EC2 module can be shared across multiple development teams and environments (dev, staging, prod).
• Simplified Collaboration: Multiple engineers can work on different modules simultaneously without causing merge conflicts in a single main.tf file.
• Abstraction: Consumers of a module only need to know what inputs (variables) to supply and what outputs to expect, without needing to understand the underlying implementation logic.
• Security & Versioning: Modules can be stored in private Git repositories, version-tagged, and locked down against unauthorized modifications.

4. Setting Up a Non-Modular Terraform Project (Day-2 Recap)

Before converting our project into a module, let's establish a standard non-modular Terraform project that utilizes providers, variables, configuration files, and outputs.

Provider Configuration (main.tf)

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = var.ami_value
  instance_type = var.instance_type_value
  subnet_id     = var.subnet_id_value
}

Variable Declarations (variables.tf)

variable "ami_value" {
  description = "Value for the AMI"
  type        = string
}

variable "instance_type_value" {
  description = "Value for the instance type"
  type        = string
}

variable "subnet_id_value" {
  description = "Value for the subnet ID"
  type        = string
}

Passing Variable Values (terraform.tfvars)

Using a .tfvars file keeps sensitive data or environment-specific configurations separate from your core codebase:

ami_value           = "ami-0c7217cdde317cfec"
instance_type_value = "t2.micro"
subnet_id_value     = "subnet-0abcd1234efgh5678"

Adding Outputs (outputs.tf)

To prevent users from having to hunt through the cloud console for provisioned details, configure output variables:

output "public_ip_address" {
  description = "Public IP address of the newly created EC2 instance"
  value       = aws_instance.example.public_ip
}

5. Step-by-Step Evolution: Converting a Project into a Module

To transform our working project into a reusable Terraform module, we restructure our workspace directories.

Step A: Create a Module Directory Structure

Move all core resource configuration files (main.tf, variables.tf, outputs.tf) inside a dedicated module directory:

Step B: The Child Module Files

Inside modules_ec2_instance/, our files remain clean and self-contained:

• main.tf: Declares the aws_instance resource.
• variables.tf: Defines ami_value, instance_type_value, and subnet_id_value.
• outputs.tf: Exposes public_ip_address.

Note: Remove terraform.tfvars from the child module. Modules should remain agnostic of hardcoded environment values; values are passed dynamically by the caller.

6. Consuming the Module from a Root Configuration

Now, any team member or project can consume our newly created EC2 module by writing a lightweight root main.tf file without rewriting the underlying resource logic.

Root main.tf Invoking the Child Module

provider "aws" {
  region = "us-east-1"
}

module "ec2_instance" {
  source              = "./modules_ec2_instance"
  ami_value           = "ami-0c7217cdde317cfec"
  instance_type_value = "t2.micro"
  subnet_id_value     = "subnet-0abcd1234efgh5678"
}

Execution Workflow

Initialize and apply the root configuration to deploy the infrastructure via the module:

# 1. Initialize working directory (downloads providers and parses local modules)
terraform init

# 2. Review execution plan
terraform plan

# 3. Apply changes
terraform apply

Root Project (main.tf)
   |
   | calls module with input variables (ami, instance_type, subnet_id)
   v
Child Module (./modules_ec2_instance)
   |
   | provisions AWS resources
   v
AWS Cloud (EC2 Instance Running)

7. Public vs. Private Terraform Registries

When building infrastructure at scale, you have two primary sources for modules:

To consume a publicly available module from the Terraform Registry, reference it directly by its registry address:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.14.0"
  
  name = "my-vpc"
  cidr = "10.0.0.0/16"
  # ... additional configuration parameters
}

8. Summary and Key Takeaways

• Monolithic Anti-Pattern: Writing all infrastructure in a single giant file leads to poor readability, lack of ownership, difficult testing, and maintenance bottlenecks.
• Modular Architecture: Terraform modules mirror the software industry's shift to microservices, providing encapsulation, reusability, and clean abstraction.
• Module Separation: A module typically consists of main.tf (resources), variables.tf (inputs), and outputs.tf (return values).
• Environment Agnosticism: Child modules should avoid hardcoded terraform.tfvars files, allowing callers to pass environment-specific values dynamically.
• Local vs. Remote Sources: Modules can be referenced locally via relative file paths (./modules/ec2) or fetched remotely from private Git repositories or public registries.
• Simplified Collaboration: Teams can scale infrastructure deployments securely by writing lightweight root configurations that consume standardized, pre-tested organizational modules.