Day 15: AWS Ultimate CI/CD Pipeline & End-to-End Demo (AWS CodePipeline & CodeDeploy)

In this session, we complete the second half of our robust AWS CI/CD pipeline by implementing AWS CodeDeploy and integrating it with AWS CodePipeline and Amazon EC2. Building directly upon our Continuous Integration (CI) foundation from Day 14, this guide bridges the gap between source code version control, automated building via AWS CodeBuild, and automated continuous delivery (CD) to production hosting servers.

1. Why Do We Need Continuous Delivery (CD)?

• The Problem with Manual Deployments: Manually SSHing into servers, pulling code, stopping old containers, and starting new ones is error-prone, slow, and impossible to scale when managing hundreds of microservices.
• The Status Quo Shift: While Day 14 focused on Continuous Integration (CI) — compiling code, running tests, and pushing Docker images to registries (e.g., Docker Hub) — the process remains incomplete without an automated Continuous Delivery (CD) mechanism to push those builds into target hosting environments.
• Role of AWS CodeDeploy: CodeDeploy acts as the automated delivery engine that pulls the validated build artifacts or container commands and safely deploys them to target infrastructure (EC2, ECS, or Lambda) without human intervention.

2. Core Mental Model: The CI/CD Orchestration Flow

Think of AWS CodePipeline as the master factory foreman (the orchestrator) that coordinates separate specialized workers:

1. Source Stage: CodeCommit or GitHub detects a new git commit.
2. Build Stage (AWS CodeBuild): Compiles the application, builds Docker images, and pushes them to Docker Hub.
3. Deployment Stage (AWS CodeDeploy): Pulls the deployment specifications (appspec.yml) and instructs the target EC2 instance to fetch and run the latest container.

3. Step 1: Setting up the Target Infrastructure (Amazon EC2)

Before CodeDeploy can push an application, it needs a target server with the proper networking, security, and agent software installed.

Launching the Ubuntu EC2 Instance

• AMI: Ubuntu (Latest LTS recommended).
• Instance Type: t2.micro (sufficient for proof-of-concept testing).
• VPC & Networking: Use the default VPC and ensure Auto-assign Public IP is enabled.
• Security Group: Open SSH (Port 22) and optionally HTTP/Application ports (e.g., Port 5000 for our Python Flask application).

The Power of AWS Resource Tags

Tags are key-value pairs assigned to AWS resources that serve two vital purposes:

1. Target Identification for CodeDeploy: Instead of hardcoding hard-to-track instance IDs, CodeDeploy uses tags (e.g., name: sample-python) to dynamically discover target servers, scaling automatically as instances are added or removed.
2. Cost Allocation & Management: Management and DevOps engineers can filter resources across S3, EC2, and EBS by project teams (e.g., project: payments) and environments (env: production).

4. Step 2: Installing and Configuring the CodeDeploy Agent

AWS CodeDeploy cannot push code directly into a bare EC2 instance without a resident listener software called the CodeDeploy Agent.

Step-by-Step Agent Installation on Ubuntu

Log into your EC2 instance via SSH and execute the following commands precisely:

# 1. Update system package lists
sudo apt update -y

# 2. Install Ruby (required for running the CodeDeploy agent) and Wget
sudo apt install ruby-full wget -y

# 3. Download the CodeDeploy agent installer script for your specific AWS Region (e.g., us-east-1)
cd /home/ubuntu
wget https://aws-codedeploy-us-east-1.s3.us-east-1.amazonaws.com/latest/install

# 4. Grant executable permissions to the installer script
chmod +x install

# 5. Execute the installer as root
sudo ./install auto

# 6. Verify and start the agent service
sudo service codedeploy-agent status
sudo service codedeploy-agent start

Note: If codedeploy-agent status returns an error stating no agent is running, execute sudo service codedeploy-agent start to initialize the host agent daemon.

5. Step 3: Configuring IAM Roles for Secure Service Communication

Security on AWS dictates that users use IAM Users, while AWS services communicate using IAM Roles.

1. Create an EC2 / CodeDeploy IAM Role:
   • Go to the IAM Console -> Roles -> Create Role.
   • Select EC2 and attach policies granting permissions for CodeDeploy and EC2 Full Access (or granular custom policies for proof-of-concept simplicity).
   • Name the role ec2-codedeploy-role.
2. Attach Role to the EC2 Instance:
   • Navigate to the EC2 Console -> Select your instance -> Actions -> Security -> Modify IAM Role -> Select ec2-codedeploy-role -> Update.
3. Restart the Agent:
   • Return to your EC2 terminal and restart the agent to pick up the new IAM permissions:
     BASH

     Copy

     sudo service codedeploy-agent restart
     

6. Step 4: Creating the CodeDeploy Application & Deployment Group

Now we register our application within the CodeDeploy service.

1. Create CodeDeploy Application:
   • Search for CodeDeploy in the AWS Console -> Click Create application.
   • Application name: sample-python-app
   • Compute platform: Choose EC2/On-premises.
2. Create Deployment Group:
   • Inside the application, click Create deployment group.
   • Deployment group name: sample-python-app-group
   • Service role: Select your ec2-codedeploy-role.
   • Deployment type: Choose In-place (for simple single-instance deployments; alternative options include Blue/Green deployments using load balancers).
   • Environment configuration: Select Amazon EC2 Instances and target your instance using its tag (name = sample-python).
   • Uncheck load balancer options for this simplified demo and click Create deployment group.

7. Step 5: Structuring the Deployment Files (appspec.yml & Scripts)

Unlike AWS CodeBuild (which allows custom build spec paths), AWS CodeDeploy strictly expects the configuration file named appspec.yml at the root directory of your GitHub repository.

The appspec.yml Blueprint

Create appspec.yml at your repository root:

version: 0.0
os: linux
files:
  - source: /
    destination: /home/ubuntu/app
hooks:
  ApplicationStop:
    - location: scripts/stopcontainer.sh
      timeout: 300
      runas: root
  AfterInstall:
    - location: scripts/startcontainer.sh
      timeout: 300
      runas: root

Container Lifecycle Scripts

Inside a scripts/ folder at your repository root, create helper shell scripts to manage container execution:

scripts/stopcontainer.sh

#!/bin/bash
# Stop and remove existing running containers to prevent port binding conflicts
CONTAINER_ID=$(sudo docker ps -q)
if [ ! -z "$CONTAINER_ID" ]; then
    sudo docker stop $CONTAINER_ID
    sudo docker rm -f $CONTAINER_ID
fi

scripts/startcontainer.sh

#!/bin/bash
# Pull the latest image from Docker Hub and run it on port 5000
sudo docker pull <your-dockerhub-username>/simple-python-app:latest
sudo docker run -d -p 5000:5000 <your-dockerhub-username>/simple-python-app:latest

8. Real-World Practical Pitfalls & Debugging Mindset

When deploying applications in production, engineers frequently run into common pitfalls. Here is how to diagnose and resolve them:

9. Complete End-to-End Flow Walkthrough

Once CodeDeploy is successfully configured, connect it to your existing AWS CodePipeline created in Day 14:

1. Add Deployment Stage to CodePipeline:
   • Go to CodePipeline -> Select your pipeline -> Click Edit.
   • Click Add stage -> Name it Deploy.
   • Add an action group: Action provider = AWS CodeDeploy, select your application name (sample-python-app) and deployment group (sample-python-app-group).
   • Save the pipeline configuration.
2. Trigger the Pipeline:
   • Push a trivial modification (e.g., adding a whitespace or updating text) to your GitHub repository.
   • Watch CodePipeline automatically trigger:
     Structure & Concept Flow

     Copy DiagramCopy

     GitHub Commit

     |

     v

     CodePipeline detects webhook change

     |

     v

     CodeBuild compiles and pushes image to Docker Hub

     |

     v

     CodeDeploy contacts EC2 CodeDeploy Agent

     |

     v

     EC2 executes stopcontainer.sh & startcontainer.sh

     |

     v

     New Python Flask Container running live on Port 5000!

      

10. Summary and Key Takeaways

• Decoupled Architecture: AWS CodeDeploy abstracts deployment logic away from build servers, communicating securely via lightweight background agents running on target EC2 instances.
• Tag-Based Target Discovery: Utilizing AWS resource tags enables CodeDeploy to dynamically target single instances or auto-scaling fleets without hardcoding IP addresses.
• Mandatory Root Structure: appspec.yml must reside strictly at the root of a GitHub repository to orchestrate pre-install, install, and post-install lifecycle hooks.
• Lifecycle Scripting: Robust CI/CD pipelines require explicit container teardown scripts (stopcontainer.sh) to prevent port collision errors during continuous updates.
• Unified Orchestration: Combining CodeCommit/GitHub, CodeBuild, and CodeDeploy inside AWS CodePipeline yields a fully automated, hands-off enterprise deployment workflow.