Log Rotation & Cleanup Scripts

This chapter covers the execution, verification, and optimization of automated deployment shell scripts. We explore how to handle errors at critical stages (such as Git cloning, package installation, and Docker builds), configure AWS security groups to expose applications, and implement alerting mechanisms like sendmail when deployments fail.

1. The Core Mental Model of Robust Deployment Scripts

In DevOps, a deployment script must be more than just a sequence of commands executed one after another. It must be resilient and self-aware. If any step in the deployment pipeline fails, the script must halt execution immediately and notify the administrator rather than continuing blindly.

Real-Life Analogy

Think of an automated car assembly line. If the engine installation step fails, the assembly line must stop immediately. Continuing to paint and polish a car without an engine is a waste of resources and hides the critical failure. Similarly, if a code dependency fails to install, building and running the Docker container is futile.

2. Step-by-Step Deployment Flow

The execution flow of our deployment script (deploy.sh) follows a strict sequential path where each step depends on the success of the previous one:

3. Exposing the Application: AWS Security Group Configuration

Once the application runs inside the Docker container on port 8000, it is isolated within the EC2 instance. To make it accessible to the public internet, we must configure the AWS Security Group.

Step-by-Step Port Configuration

1. Navigate to your AWS EC2 Dashboard.
2. Select your running EC2 Instance.
3. Click on the Security tab and select the associated Security Group.
4. Click Edit inbound rules.
5. Add a new rule with the following parameters:

6. Save the rules and access the application in your browser using:
   CODE

   Copy

   http://<EC2_Public_IP>:8000
   

4. The Idempotency & Redundancy Issue

When we re-run the deployment script, it executes successfully, but it introduces several inefficiencies:

• Redundant Service Actions: It re-enables Nginx (systemctl enable nginx) even if it is already active.
• Unnecessary Restarts: It restarts the Docker daemon and rebuilds the image from scratch, causing temporary application downtime.

Concept: Idempotency

Idempotency is the property of a script or operation where running it multiple times yields the exact same system state without causing unintended side effects or redundant resource consumption. A production-grade script should check if a service is already running before attempting to start or enable it.

5. Enhancing Script Robustness with Error Handling & Alerts

To prevent silent failures during the container run phase, we can chain commands using Bash logical operators (||) and integrate system alerting utilities like sendmail.

The || (OR) Operator

In Bash, command1 || command2 ensures that command2 runs only if command1 exits with a non-zero (failure) status code.

Integrating sendmail

If the Docker container fails to start, the script should catch the error, log it, and send an email notification to the administrator:

docker run -d -p 8000:8000 --name my-app notes:latest || {
    echo "Deployment Failed! Mailing the administrator..."
    echo "Subject: Deployment Failure Alert" | sendmail admin@example.com
    exit 1
}

6. Code Evolution: Naive vs. Production-Ready Script

Naive Script (No Error Handling or Idempotency Checks)

#!/bin/bash
# This script runs blindly and restarts services unnecessarily

git clone https://github.com/example/notes-app.git
cd notes-app
npm install
systemctl enable nginx
systemctl start nginx
docker build -t notes:latest .
docker run -d -p 8000:8000 --name my-app notes:latest

Production-Ready Script (With Error Handling, Alerts, and Checks)

#!/bin/bash

# Exit immediately if any command exits with a non-zero status
set -e

echo "Starting Deployment..."

# 1. Clone Repository with Error Handling
if [ ! -d "notes-app" ]; then
    git clone https://github.com/example/notes-app.git || { echo "Git clone failed"; exit 1; }
else
    echo "Repository directory already exists. Pulling latest changes..."
    cd notes-app && git pull || { echo "Git pull failed"; exit 1; }
    cd ..
fi

cd notes-app

# 2. Install Dependencies
echo "Installing dependencies..."
npm install || { echo "Dependency installation failed"; exit 1; }

# 3. Idempotent Nginx Configuration
if ! systemctl is-active --quiet nginx; then
    echo "Starting Nginx..."
    sudo systemctl enable nginx
    sudo systemctl start nginx
else
    echo "Nginx is already running."
fi

# 4. Build Docker Image
echo "Building Docker image..."
docker build -t notes:latest . || { echo "Docker build failed"; exit 1; }

# 5. Run Container with Port Conflict Resolution & Mail Alerts
echo "Deploying container..."
docker rm -f my-app 2>/dev/null || true

docker run -d -p 8000:8000 --name my-app notes:latest || {
    echo "Deployment Failed! Mailing the administrator..."
    echo -e "Subject: DEPLOYMENT FAILURE\n\nDocker container failed to start on port 8000." | sendmail admin@example.com
    exit 1
}

echo "Deployment completed successfully!"

7. Summary and Key Takeaways

• Graceful Error Handling: Always design scripts to catch failures at critical junctions (Git clone, package install, Docker run) instead of letting them fail silently.
• AWS Port Management: Applications running inside Docker containers on specific ports (e.g., 8000) require explicit inbound rule configurations in AWS Security Groups to be publicly accessible.
• Security Best Practices: When opening ports in AWS, restricting the source to My IP is highly recommended for development environments to prevent unauthorized access.
• Idempotency is Key: Production scripts should check the current state of services (e.g., using systemctl is-active) before executing start or enable commands to avoid redundant operations.
• Proactive Alerting: Integrating notification utilities like sendmail using Bash logical operators (||) ensures that system administrators are immediately alerted in the event of a production failure.
• Container Lifecycle Management: Before running a new container with the same name, ensure old instances are stopped and removed (docker rm -f) to prevent name and port conflicts.