AWS Zero to Hero: Beginner-Level Shell Scripting Project for DevOps

This chapter focuses on building a practical, production-grade shell scripting automation project commonly used by DevOps engineers to audit active cloud resources. By writing a script that queries AWS services across specified regions, you will learn how to enforce input validation, handle environment dependencies, implement clean control flows, and apply industry-standard security permissions.

1. The DevOps Problem Statement: Why Automate Resource Auditing?

In modern cloud infrastructure, cost optimization and resource visibility are critical operational pillars. Organizations often run hundreds of services across multiple AWS accounts and regions. Without automated tracking, orphaned resources—such as unattached Elastic Block Store (EBS) volumes, idle Elastic Compute Cloud (EC2) instances, or forgotten Simple Storage Service (S3) buckets—can silently inflate the monthly cloud bill.

Real-World Interview Context

Interviewers rarely ask candidates to recite basic shell commands (e.g., "How do you list running processes?"). Instead, they focus on practical application:

• “What kind of shell scripts have you written to automate day-to-day tasks in your current organization?”
• “How do you ensure your scripts are secure and robust against user errors?”

By implementing this resource-listing project, you gain a concrete, production-ready automation example that demonstrates your understanding of shell scripting, API integration, and cloud governance.

2. Core Mental Model & Architecture Flow

To interact with AWS, a script must communicate with the AWS API. While you could write raw HTTP requests using curl, doing so requires manually constructing complex authentication headers and signing requests with HMAC-SHA256.

To simplify this, we use the AWS Command Line Interface (CLI) as an intermediary abstraction layer. The shell script calls the AWS CLI, which handles authentication, payload formatting, and API communication on our behalf.

Execution Workflow

3. Designing the Script: Header Metadata & Best Practices

A production-grade script must be self-documenting. When other developers or system administrators run your script, they should immediately understand its purpose, author, version, and usage without reading the entire codebase.

Key Header Components

1. Shebang (#!/bin/bash): Instructs the operating system's program loader to execute the script using the Bash shell.
2. Description: A clear statement of what the script does.
3. Metadata: Author name, team ownership, and version control.
4. Usage Instructions: Explicit examples showing how to pass command-line arguments.

Header Implementation

#!/bin/bash

###############################################################################
# Author: Abhishek / DevOps Team
# Version: v0.0.1
#
# Description: This script lists active AWS resources in a specified region.
#
# Supported Services:
#   - ec2: EC2 Instances
#   - s3: S3 Buckets
#   - rds: RDS Databases
#   - dynamodb: DynamoDB Tables
#   - ebs: EBS Volumes
#   - elb: Elastic Load Balancers
#   - vpc: VPC Resources
#   - cloudformation: CloudFormation Stacks
#   - iam: IAM Users/Roles
#
# Usage:
#   ./AWS_resource_list.sh <region> <service>
#
# Example:
#   ./AWS_resource_list.sh us-east-1 ec2
###############################################################################

4. Step 1: Input Validation & Argument Parsing

When a script accepts command-line arguments, you must validate the inputs before executing any core logic. If a user runs the script with missing parameters, the script should fail gracefully with a clear usage message rather than throwing cryptic API errors.

In Bash:

• $# is a special variable that stores the number of arguments passed to the script.
• $0 represents the name of the script itself.
• $1 and $2 represent the first and second arguments, respectively.

Validation Logic

# Check if the required number of arguments (2) are passed
if [ $# -ne 2 ]; then
    echo "Error: Invalid number of arguments."
    echo "Usage: $0 <region> <service>"
    echo "Example: $0 us-east-1 ec2"
    exit 1
fi

# Assign command-line arguments to descriptive variables
AWS_REGION=$1
AWS_SERVICE=$2

5. Step 2: Environment Verification (AWS CLI Installation & Configuration Checks)

Before invoking AWS commands, the script must verify that:

1. The AWS CLI binary is installed and accessible in the system's PATH.
2. The AWS CLI has been configured with valid credentials (which creates the ~/.aws directory).

Verification Logic

# Check if AWS CLI is installed
if ! command -v aws &> /dev/null; then
    echo "Error: AWS CLI is not installed. Please install it before running this script."
    exit 1
fi

# Check if AWS CLI is configured by verifying the existence of the credentials directory
if [ ! -d "$HOME/.aws" ]; then
    echo "Error: AWS CLI is not configured. Please run 'aws configure' to set up credentials."
    exit 1
fi

6. Step 3: Implementing the Switch Case for Multi-Service Support

To execute different AWS CLI commands based on the user's service input, we must implement conditional routing. While an if-else block works, a switch case statement is cleaner, more readable, and easier to maintain as you add support for more services.

Algorithmic Complexity & Readability Comparison

Switch Case Syntax in Bash

A switch case block begins with case <variable> in and ends with esac (which is "case" spelled backward). Each pattern is terminated with a double semicolon ;;.

case $AWS_SERVICE in
    ec2)
        echo "Listing EC2 instances in region $AWS_REGION..."
        aws ec2 describe-instances --region "$AWS_REGION"
        ;;
    s3)
        echo "Listing S3 buckets..."
        aws s3 api list-buckets
        ;;
    *)
        echo "Error: Unsupported service '$AWS_SERVICE'."
        echo "Supported services are: ec2, s3, rds, dynamodb, ebs, elb, vpc, cloudformation, iam"
        exit 1
        ;;
esac

7. The Complete Production-Ready Shell Script

Below is the complete, integrated script containing the header, input validation, environment checks, and the switch-case routing logic.

#!/bin/bash

###############################################################################
# Author: Abhishek / DevOps Team
# Version: v1.0.0
#
# Description: This script lists active AWS resources in a specified region.
#
# Supported Services:
#   - ec2, s3, rds, dynamodb, ebs, elb, vpc, cloudformation, iam
#
# Usage:
#   ./AWS_resource_list.sh <region> <service>
###############################################################################

# 1. Input Validation
if [ $# -ne 2 ]; then
    echo "Error: Invalid number of arguments."
    echo "Usage: $0 <region> <service>"
    echo "Example: $0 us-east-1 ec2"
    exit 1
fi

AWS_REGION=$1
AWS_SERVICE=$2

# 2. Check if AWS CLI is installed
if ! command -v aws &> /dev/null; then
    echo "Error: AWS CLI is not installed. Please install it before running this script."
    exit 1
fi

# 3. Check if AWS CLI is configured
if [ ! -d "$HOME/.aws" ]; then
    echo "Error: AWS CLI is not configured. Please run 'aws configure' to set up credentials."
    exit 1
fi

# 4. Route execution based on service argument
case ${AWS_SERVICE,,} in # ,, converts input to lowercase to handle case insensitivity
    ec2)
        echo "Querying active EC2 instances in $AWS_REGION..."
        aws ec2 describe-instances --region "$AWS_REGION" --query 'Reservations[*].Instances[*].{ID:InstanceId,Type:InstanceType,State:State.Name}' --output table
        ;;
    s3)
        echo "Querying S3 buckets..."
        aws s3 api list-buckets --query 'Buckets[*].Name' --output table
        ;;
    rds)
        echo "Querying RDS instances in $AWS_REGION..."
        aws rds describe-db-instances --region "$AWS_REGION" --query 'DBInstances[*].{DBInstanceIdentifier:DBInstanceIdentifier,Engine:Engine,Status:DBInstanceStatus}' --output table
        ;;
    dynamodb)
        echo "Querying DynamoDB tables in $AWS_REGION..."
        aws dynamodb list-tables --region "$AWS_REGION" --output table
        ;;
    ebs)
        echo "Querying EBS volumes in $AWS_REGION..."
        aws ec2 describe-volumes --region "$AWS_REGION" --query 'Volumes[*].{ID:VolumeId,Size:Size,State:State}' --output table
        ;;
    elb)
        echo "Querying Elastic Load Balancers in $AWS_REGION..."
        aws elbv2 describe-load-balancers --region "$AWS_REGION" --query 'LoadBalancers[*].{Name:LoadBalancerName,DNS:DNSName,Type:Type}' --output table
        ;;
    vpc)
        echo "Querying VPCs in $AWS_REGION..."
        aws ec2 describe-vpcs --region "$AWS_REGION" --query 'Vpcs[*].{ID:VpcId,CidrBlock:CidrBlock,IsDefault:IsDefault}' --output table
        ;;
    cloudformation)
        echo "Querying CloudFormation stacks in $AWS_REGION..."
        aws cloudformation list-stacks --region "$AWS_REGION" --stack-status-filter CREATE_COMPLETE UPDATE_COMPLETE --query 'StackSummaries[*].{Name:StackName,Status:StackStatus}' --output table
        ;;
    iam)
        echo "Querying IAM users..."
        aws iam list-users --query 'Users[*].{UserName:UserName,UserId:UserId}' --output table
        ;;
    *)
        echo "Error: Unsupported service '$AWS_SERVICE'."
        echo "Supported services: ec2, s3, rds, dynamodb, ebs, elb, vpc, cloudformation, iam"
        exit 1
        ;;
esac

8. Setting Up the Execution Environment on AWS EC2

To run and test this script, we will launch an Ubuntu EC2 instance on AWS and connect to it.

Step-by-Step EC2 Launch Process

1. Navigate to the AWS Console and select your closest region (e.g., ap-south-1 for India).
2. Go to the EC2 Dashboard and click Launch Instance.
3. Name: Scripting-Test-Instance.
4. OS Image: Select Ubuntu (LTS version).
5. Instance Type: Select t2.micro (Free Tier eligible).
6. Key Pair: Select an existing key pair or create a new one (e.g., devops-key.pem).
7. Click Launch Instance.

Connecting to the Instance via SSH

Open your terminal (Git Bash, PowerShell, or macOS Terminal), navigate to your downloads folder containing the .pem key, and run:

# Set secure permissions on the private key
chmod 400 devops-key.pem

# Connect to the EC2 instance
ssh -i devops-key.pem ubuntu@<YOUR_EC2_PUBLIC_IP>

9. Installing and Configuring the AWS CLI

Once logged into your EC2 instance, you must install the AWS CLI and configure it with programmatic access credentials.

Step 1: Install System Dependencies & AWS CLI

Run the following commands to update your package manager, install unzip, and download the AWS CLI installation package:

# Update package lists
sudo apt update -y

# Install unzip utility
sudo apt install unzip -y

# Download the AWS CLI v2 zip package
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

# Unzip the package
unzip awscliv2.zip

# Run the installer
sudo ./aws/install

# Verify installation
aws --version

Step 2: Configure AWS Credentials

To allow the CLI to authenticate requests, run the configuration wizard:

aws configure

You will be prompted to enter your credentials:

• AWS Access Key ID: AKIAIOSFODNN7EXAMPLE
• AWS Secret Access Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
• Default region name: ap-south-1 (or your preferred region)
• Default output format: json

Security Note: Never use your root AWS account credentials for daily CLI operations. Instead, create an IAM User with the PowerUserAccess or ReadOnlyAccess policy attached, and use those credentials.

10. Executing and Testing the Script

Now that the environment is configured, we can create, authorize, and run our script.

Step 1: Create the Script File

Use a terminal text editor like nano to create the file:

nano AWS_resource_list.sh

Paste the complete script code from Section 7 into the editor, save the file (Ctrl+O, Enter), and exit (Ctrl+X).

Step 2: Test Validation Logic

Run the script with missing arguments to verify that validation works:

./AWS_resource_list.sh

Expected Output:

Error: Invalid number of arguments.
Usage: ./AWS_resource_list.sh <region> <service>
Example: ./AWS_resource_list.sh us-east-1 ec2

Step 3: Test Resource Queries

Run the script with valid parameters to query active resources:

# Query EC2 instances in ap-south-1
bash AWS_resource_list.sh ap-south-1 ec2

# Query S3 buckets
bash AWS_resource_list.sh ap-south-1 s3

11. Security Best Practices: File Permissions & Access Control

In a shared Linux environment, protecting your scripts from unauthorized modification or viewing is a critical security practice. This is especially true if your scripts contain internal system paths, logic, or configurations.

We use the chmod utility to set precise access permissions.

Understanding Octal Permissions

Linux permissions are calculated using three octal digits representing Owner, Group, and Others:

• 7 (Read + Write + Execute): $4 + 2 + 1 = 7$
• 5 (Read + Execute): $4 + 0 + 1 = 5$
• 1 (Execute Only): $0 + 0 + 1 = 1$

Recommended Permissions for Production Scripts

chmod 751 AWS_resource_list.sh

This permission scheme breaks down as follows:

  7 (Owner)       5 (Group)       1 (Others)
  [rwx]           [r-x]           [--x]
  Read, Write,    Read & Execute  Execute Only
  & Execute       Only            (Cannot view code)

By setting permissions to 751 (or 771 depending on team requirements), you allow external users or automated service accounts to run the script without granting them permission to read or modify its source code.

12. Summary and Key Takeaways

• Cost Optimization: Automating cloud resource auditing helps organizations identify and clean up orphaned resources, directly reducing cloud spend.
• Input Validation: Always validate command-line arguments at the very beginning of your script using $# to prevent runtime execution failures.
• Environment Checks: Verifying dependencies (like checking if the AWS CLI binary exists and is configured) ensures your script fails gracefully with clear instructions.
• Switch Case vs. If-Else: Use switch case blocks instead of nested if-else statements when routing logic based on a single variable. This improves code readability and maintainability.
• AWS CLI Integration: Leveraging the AWS CLI as an abstraction layer avoids the complexity of writing raw API requests and manually signing authentication headers.
• Least Privilege Access: Always run CLI scripts using credentials from an IAM user with restricted permissions rather than the AWS root account.
• File Security: Apply strict Linux file permissions (such as chmod 751) to prevent unauthorized users from viewing or modifying your automation scripts.