Day-06 | Create Resources on AWS using Ansible | Ansible Variables & Vault Demo

This chapter covers the core concepts of Ansible Collections, Ansible Variables, and Ansible Vault. We will explore how to transition from managing traditional virtual machines via SSH to provisioning cloud infrastructure on AWS using API-driven collections. Additionally, we will examine how to secure sensitive credentials using Ansible Vault and manage configuration flexibility through Ansible's variable precedence hierarchy.

1. The Paradigm Shift: SSH-Based vs. API-Based Automation

In previous sessions, Ansible was used to manage target operating systems (such as EC2 instances) by establishing SSH connections, copying Python modules to the target, and executing them locally on the managed nodes.

However, when provisioning cloud infrastructure (e.g., AWS EC2 instances, S3 buckets, RDS databases) or managing network appliances (e.g., Cisco switches, F5 BIG-IP load balancers), you cannot SSH directly into the cloud provider's control plane or SaaS platform. Instead, you must interact with their exposed web APIs.

Comparison of Execution Paradigms

2. Architecture and Flow: How Ansible Interacts with Cloud APIs

When executing cloud provisioning playbooks, Ansible runs the tasks locally on the control node. The control node uses a specialized Python SDK to translate your declarative YAML configuration into API calls directed at the cloud provider.

3. Prerequisites: Installing the Amazon AWS Collection and Boto3

To interact with AWS, you must install the official Ansible AWS collection and its underlying Python dependency, boto3, on your control node.

Step 1: Install the Amazon AWS Collection

Ansible Collections group related roles, modules, and plugins. Use the ansible-galaxy command-line tool to pull down the collection:

ansible-galaxy collection install amazon.aws

Note: If you need to force-update an existing collection to the latest version, append the --force flag:

BASH

Copy

ansible-galaxy collection install amazon.aws --force

Step 2: Install the Boto3 SDK

The amazon.aws collection relies on boto3 (the AWS SDK for Python) to authenticate and communicate with AWS APIs. Install it on your control node using your Python package manager:

pip install boto3

Note: Depending on your operating system and Python environment, you may need to use pip3 or pipx to prevent system-level package conflicts.

4. Step-by-Step: Initializing the EC2 Provisioning Role

To maintain clean, production-grade code, we will package our EC2 provisioning logic inside an Ansible Role.

Step 1: Create the Project Directory Structure

Create a dedicated directory for your AWS playbooks and navigate into it:

mkdir -p ansible-aws-demo/roles
cd ansible-aws-demo

Step 2: Initialize the EC2 Role

Use ansible-galaxy to generate the standard directory structure for a role named ec2:

ansible-galaxy role init roles/ec2

This command creates the standard Ansible role directory layout:

5. Securing Credentials with Ansible Vault

Never hardcode sensitive credentials like AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY directly into your playbooks or version control systems. Ansible Vault provides a built-in mechanism to encrypt variables and files.

Step 1: Generate a Vault Password File

To avoid entering the vault password manually during every execution, store a secure, random password in a local file. Ensure this file is added to your .gitignore so it is never committed.

openssl rand -base64 32 > vault.pass
chmod 600 vault.pass

Step 2: Create an Encrypted Vault File

Create an encrypted file named vault.yml to store your AWS credentials:

ansible-vault create --vault-password-file vault.pass roles/ec2/vars/vault.yml

This command opens your default terminal editor (e.g., vim or nano). Add your AWS credentials in standard YAML format:

vault_aws_access_key: "AKIAIOSFODNN7EXAMPLE"
vault_aws_secret_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Save and close the file. If you inspect roles/ec2/vars/vault.yml using cat, you will see encrypted ciphertext:

$ANSIBLE_VAULT;1.1;AES256
3635363231373737616263643030313233343536373839306162636465666768...

6. Writing the Naive (Hardcoded) Playbook

To understand the value of variables, let's first look at a naive, hardcoded task file. This approach is rigid because changing the instance type or region requires modifying the core code.

Write the following configuration to roles/ec2/tasks/main.yml:

---
# roles/ec2/tasks/main.yml (Naive, Hardcoded Version)
- name: Start an EC2 Instance with Public IP
  amazon.aws.ec2_instance:
    name: "ansible-production-web"
    key_name: "my-aws-key"
    instance_type: "t2.micro"
    image_id: "ami-0c7217cdde317cfec" # Ubuntu 22.04 LTS in us-east-1
    region: "us-east-1"
    aws_access_key: "{{ vault_aws_access_key }}"
    aws_secret_key: "{{ vault_aws_secret_key }}"
    network:
      assign_public_ip: true

The Problems with Hardcoding:

1. Lack of Reusability: If another team needs a t2.medium instance instead of a t2.micro, they must modify the task file directly.
2. Environment Lock-in: The AMI ID and region are hardcoded to us-east-1. Running this in us-west-2 will fail because AMI IDs are region-specific.
3. Risk of Code Regression: Allowing users to modify core task files to change configurations increases the risk of introducing syntax errors.

7. Refactoring with Variables and Jinja2 Templating

To make the role reusable, we can parameterize the configuration using variables. Ansible uses Jinja2 templating syntax to reference variables.

Jinja2 Syntax Rules in Ansible:

• Variables must be enclosed in double curly braces: {{ variable_name }}.
• If a variable reference starts a YAML value, the entire expression must be enclosed in quotes to prevent YAML parser errors:
  YAML

  Copy

  # Correct:
  instance_type: "{{ ec2_instance_type }}"
  
  # Incorrect (Will throw a YAML syntax error):
  instance_type: {{ ec2_instance_type }}
  

Refactored Task File

Update roles/ec2/tasks/main.yml to use variables:

---
# roles/ec2/tasks/main.yml (Refactored, Parameterized Version)
- name: Start an EC2 Instance with Public IP
  amazon.aws.ec2_instance:
    name: "{{ ec2_instance_name }}"
    key_name: "{{ ec2_key_name }}"
    instance_type: "{{ ec2_instance_type }}"
    image_id: "{{ ec2_ami_id }}"
    region: "{{ ec2_region }}"
    aws_access_key: "{{ vault_aws_access_key }}"
    aws_secret_key: "{{ vault_aws_secret_key }}"
    network:
      assign_public_ip: true

8. Deep Dive: Ansible Variable Precedence

Ansible allows you to declare variables in multiple places (up to 22 different locations). This flexibility requires a strict order of priority (precedence) to resolve conflicts when the same variable is defined in multiple locations.

The Key Variable Locations (Lowest to Highest Precedence)

Detailed Precedence Table

9. Practical Demonstration: Overriding Variables in Action

Let's configure our role to demonstrate how variable precedence works in practice.

Step 1: Define Default Variables (Lowest Precedence)

Define your default values in roles/ec2/defaults/main.yml. These values are used if no other overrides are provided.

---
# roles/ec2/defaults/main.yml
ec2_instance_name: "ansible-default-web"
ec2_key_name: "my-aws-key"
ec2_instance_type: "t2.micro"
ec2_ami_id: "ami-0c7217cdde317cfec" # Ubuntu 22.04 LTS
ec2_region: "us-east-1"

Step 2: Define Role Variables (Higher Precedence)

Define variables in roles/ec2/vars/main.yml. These values will override the defaults.

---
# roles/ec2/vars/main.yml
# This overrides the t2.micro defined in defaults/main.yml
ec2_instance_type: "t2.medium"

Step 3: Create the Master Playbook

Create a playbook named deploy_ec2.yml in the root directory:

---
# deploy_ec2.yml
- name: Provision AWS Infrastructure
  hosts: localhost
  connection: local
  gather_facts: false
  vars_files:
    - roles/ec2/vars/vault.yml
  roles:
    - ec2

Step 4: Execute and Observe Precedence

Scenario A: Running with Role Variables

If you run the playbook now, Ansible will merge the defaults and role variables. Since vars/main.yml has higher precedence than defaults/main.yml, Ansible will provision a t2.medium instance.

ansible-playbook -i localhost, deploy_ec2.yml --vault-password-file vault.pass

Scenario B: Overriding with Extra Vars (Highest Precedence)

To override both the default (t2.micro) and the role variable (t2.medium) at runtime, pass the variable via the command line using the -e flag:

ansible-playbook -i localhost, deploy_ec2.yml \
  --vault-password-file vault.pass \
  -e "ec2_instance_type=t2.large"

Ansible will bypass all file-based declarations and provision a t2.large instance.

10. Group Variables: Segmenting Configurations by Inventory Groups

Group variables allow you to apply different configurations to different sets of servers without modifying your roles or playbooks.

Real-World Analogy

Think of a restaurant kitchen. The prep station and the grill station both need knives and cutting boards, but the prep station needs vegetable peelers while the grill station needs meat thermometers. Instead of giving every station every tool, you assign tools based on the station's function. Group variables work the same way by assigning specific configurations to specific groups of servers.

Directory Structure for Group Variables

To use group variables, create a directory named group_vars in the same directory as your inventory file:

Example Inventory File

[app]
webserver1.example.com
webserver2.example.com

[db]
dbserver1.example.com

Defining Group-Specific Variables

For the app group, create group_vars/app.yml:

---
# group_vars/app.yml
app_port: 8080
env_db_password: "app-db-password"

For the db group, create group_vars/db.yml:

---
# group_vars/db.yml
db_port: 5432
max_connections: 100

When Ansible runs a task against a host in the app group, it automatically loads the variables from group_vars/app.yml. When running against a host in the db group, it loads the variables from group_vars/db.yml.

11. Real-World Debugging and Execution Workflows

When working with Ansible Vault, variables, and cloud collections, you may encounter several common errors.

Error 1: Undefined Variable Error

fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ec2_instance_type' is undefined"}

• Why it happens: The variable was referenced in the task using {{ ec2_instance_type }} but was not declared in any of the active variable files.
• How to debug:
  1. Check for typos in your variable names.
  2. Verify that the file containing the variable (e.g., defaults/main.yml) is in the correct directory.
  3. Ensure the role is correctly loaded in your playbook.

Error 2: Missing Vault Password

fatal: [localhost]: FAILED! => {"msg": "Attempting to decrypt but no vault secrets found"}

• Why it happens: Your playbook references an encrypted vault file, but you did not provide the vault password during execution.
• How to debug: Always append the --vault-password-file flag to your execution command:
  BASH

  Copy

  ansible-playbook -i localhost, deploy_ec2.yml --vault-password-file vault.pass
  

Error 3: Boto3 Missing on Control Node

fatal: [localhost]: FAILED! => {"changed": false, "msg": "The Python guidelines for AWS require boto3 and botocore."}

• Why it happens: The amazon.aws collection is executing locally on your control node, but the required Python libraries are missing.
• How to debug: Run pip install boto3 botocore on your control node. If you are using a virtual environment, ensure it is activated before running the playbook.

Summary and Key Takeaways

• API-Driven Automation: Ansible uses collections to manage cloud platforms and network appliances. These modules run locally on the control node and interact with target platforms via APIs rather than SSH.
• Ansible Collections: Collections are distribution formats for Ansible content. You can install them from Ansible Galaxy using the ansible-galaxy collection install command.
• Ansible Vault: Vault secures sensitive data by encrypting files with AES-256 encryption. You can pass vault passwords to playbooks using the --vault-password-file flag.
• Jinja2 Templating: Ansible uses Jinja2 syntax ({{ variable_name }}) to reference variables. Remember to wrap the entire expression in quotes if it starts a YAML value.
• Variable Precedence: Ansible resolves variable conflicts using a strict hierarchy. defaults/main.yml has the lowest priority, vars/main.yml has a higher priority, and command-line extra vars (-e) have the highest priority.
• Group Variables: The group_vars directory allows you to define group-specific configurations based on your inventory groups, keeping your playbooks clean and modular.