AWS CLI Configuration and Automated EC2 Provisioning via Shell Scripting

This chapter covers the end-to-end process of configuring the AWS Command Line Interface (CLI) with programmatic IAM credentials and writing a robust shell script to automate EC2 instance creation. You will learn how to resolve common authentication errors, map essential AWS infrastructure parameters (such as AMI IDs, Subnets, and Security Groups), and implement a polling loop to track instance startup status.

1. The Missing Link: Why AWS CLI Needs IAM Credentials

Installing the AWS CLI binary (typically located at /usr/local/bin/aws) is only the first step. By default, the CLI is an unauthenticated client. If you attempt to run any command to provision or manage resources, AWS will reject the request because it lacks identity and permission context.

Real-Life Analogy

Think of the AWS CLI as a physical keycard reader installed on a secure office door. Having the reader installed (AWS CLI binary) does not let you inside. You must program a valid keycard (IAM Access Keys) with the correct clearance level (IAM Policies) to unlock the door and access the rooms (AWS Services).

To bridge this gap, you must establish secure programmatic access using AWS Identity and Access Management (IAM).

2. Step-by-Step: Creating a Programmatic IAM User in AWS

To allow your shell script to interact with AWS, you must create a dedicated IAM user with programmatic access and attach the minimum required permissions.

Step 1: Create the IAM User

1. Navigate to the IAM Console in AWS.
2. Click Users -> Create User.
3. Set the User Name to shell-admin.
4. Do not grant this user AWS Management Console access (they only require CLI/API access). Click Next.

Step 2: Attach Permissions (Least Privilege Principle)

1. Select Attach policies directly.
2. Search for and select AmazonEC2FullAccess. This grants the user permission to launch, stop, and terminate EC2 instances.

   Note: If your script needs to manage other resources in the future (such as S3 buckets or RDS databases), you must return to IAM and attach the corresponding policies (e.g., AmazonS3FullAccess).

3. Click Next and then Create User.

Step 3: Generate Programmatic Access Keys

1. Click on the newly created shell-admin user.
2. Go to the Security credentials tab.
3. Scroll down to Access keys and click Create access key.
4. Select Command Line Interface (CLI) as the use case.
5. Acknowledge the recommendations, click Next, and click Create access key.
6. Crucial Step: Copy both the Access Key ID and the Secret Access Key. Store them securely; the secret key will not be shown again.

3. Configuring the AWS CLI (aws configure)

Once you have your access keys, you must configure your local environment to use them.

Run the following interactive command in your terminal:

aws configure

Provide the requested inputs as shown below:

AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-2
Default output format [None]: 

Configuration Parameters Explained:

• AWS Access Key ID & Secret Access Key: The credentials generated in the previous step.
• Default region name: The AWS region where your resources will be provisioned by default (e.g., us-east-2 for US East (Ohio)).
• Default output format: Press Enter to leave it as default (json).

4. Analyzing the Naive/Broken Shell Script

If you run a naive EC2 creation script without specifying the required infrastructure parameters, the AWS API will reject the request.

The Error

When executing ./create-ec2-instance.sh with empty parameters, you will encounter the following error:

Creating EC2 instance...
An error occurred (MissingParameter) when calling the RunInstances operation: 
The request must contain the parameter ImageId

Why This Fails

An EC2 instance cannot be created in a vacuum. AWS requires specific parameters to know what operating system to install, where to place it network-wise, and how to secure it.

5. Gathering AWS Infrastructure Parameters

Before updating your shell script, you must collect five essential parameters from your AWS Console:

6. The Complete, Production-Ready Shell Script

Below is the complete, decoupled shell script. It defines the infrastructure parameters as variables at the top, executes the AWS CLI command to provision the instance, and runs a polling loop that waits until the instance transitions to the running state.

#!/bin/bash
# ==============================================================================
# Script Name:  create-ec2-instance.sh
# Description:  Automates the creation of an AWS EC2 instance and waits for it
#               to reach the "running" state.
# ==============================================================================

set -euo pipefail

# --- Configuration Variables ---
AMI_ID="ami-09040d770ffe2224f"  # Replace with your target Ubuntu/Amazon Linux AMI ID
INSTANCE_TYPE="t2.micro"
KEY_NAME="shell-scripting-for-devops-key"
SUBNET_ID="subnet-0b6b215e4f8901abc"
SECURITY_GROUP_ID="sg-0123456789abcdef0"

echo "========================================="
echo "Starting EC2 Instance Provisioning Workflow"
echo "========================================="

# 1. Launch the EC2 Instance
echo "Sending launch request to AWS EC2..."
INSTANCE_INFO=$(aws ec2 run-instances \
    --image-id "$AMI_ID" \
    --instance-type "$INSTANCE_TYPE" \
    --key-name "$KEY_NAME" \
    --subnet-id "$SUBNET_ID" \
    --security-group-ids "$SECURITY_GROUP_ID" \
    --output json)

# 2. Extract Instance ID using jq
INSTANCE_ID=$(echo "$INSTANCE_INFO" | grep -o '"InstanceId": "[^"]*' | grep -o '[^"]*$')
echo "Instance successfully initiated! Instance ID: $INSTANCE_ID"

# 3. Polling Loop: Wait for the instance to be in 'running' state
echo "Waiting for instance $INSTANCE_ID to enter 'running' state..."

while true; do
    # Query the current state of the instance
    CURRENT_STATE=$(aws ec2 describe-instances \
        --instance-ids "$INSTANCE_ID" \
        --query "Reservations[0].Instances[0].State.Name" \
        --output text)

    echo "Current State: $CURRENT_STATE"

    if [ "$CURRENT_STATE" = "running" ]; then
        echo "Success! Instance $INSTANCE_ID is now up and running."
        break
    fi

    echo "Instance is not ready yet. Sleeping for 10 seconds..."
    sleep 10
done

echo "========================================="
echo "Provisioning completed successfully!"
echo "========================================="

7. Execution Flow and Verification

The following workflow diagram illustrates how the shell script interacts with the AWS API to provision and monitor the resource:

Verification Steps:

1. Save the script using Vim: Press Esc, type :wq, and press Enter.
2. Make the script executable:
   BASH

   Copy

   chmod +x create-ec2-instance.sh
   

3. Run the script:
   BASH

   Copy

   ./create-ec2-instance.sh
   

4. Open your AWS Console under EC2 -> Instances to verify that a new t2.micro instance is initializing and matches the configuration parameters specified in your script.

Summary and Key Takeaways

• AWS CLI Authentication: The AWS CLI binary requires programmatic credentials (Access Key ID and Secret Access Key) configured via aws configure to interact with your AWS account.
• IAM Security Best Practices: Create dedicated IAM users (e.g., shell-admin) with programmatic access only, avoiding the use of the root account for automation.
• Least Privilege Principle: Assign only the necessary permissions (such as AmazonEC2FullAccess) required for the script to execute its tasks.
• Mandatory EC2 Parameters: Launching an EC2 instance programmatically requires five core parameters: AMI ID, Instance Type, Key Pair Name, Subnet ID, and Security Group ID.
• State Polling: Because AWS resource provisioning is asynchronous, scripts should implement a while loop with a sleep interval to poll the resource state (using describe-instances) until it reaches the desired state (running).
• Error Handling: Always validate that variables are populated before executing CLI commands to avoid API errors like MissingParameter.