Day 3: AWS EC2 Deep Dive and Jenkins Deployment

Amazon Elastic Compute Cloud (Amazon EC2) forms the backbone of AWS infrastructure, offering resizable, secure virtual computing capacity in the cloud. This guide covers the theoretical architecture of EC2—including virtualization, instance categorization, global infrastructure boundaries—and a practical hands-on deployment of a production automation server (Jenkins) on an Ubuntu instance, resolving standard real-world access and networking bottlenecks.

1. Understanding EC2: The Core Mental Model

The acronym EC2 stands for Elastic Compute Cloud ($E \cdot C^2$):

Fundamentally, provisioning an EC2 instance means requesting a virtual server carved out of AWS's physical enterprise hardware.

Real-Life Analogy: The Shared Physical Host

Consider your personal laptop. It contains physical compute hardware: a multi-core processor, RAM, and an SSD. Only one person uses it at a time. If three colleagues need dedicated environments on that same laptop, you install a hypervisor (such as VirtualBox or VMware) to slice the underlying physical hardware into isolated, software-defined environments called Virtual Machines (VMs).

AWS executes this model at a massive planetary scale. AWS buys high-capacity physical blade servers from enterprise vendors, deploys high-performance hypervisors, and provides isolated virtual slices (EC2 instances) upon API request.

2. Why Choose EC2? On-Premises Infrastructure vs. Public Cloud

Managing compute capacity in-house creates severe operational friction compared to provisioning virtual servers via a cloud provider.

1. Zero Hardware Maintenance Burden

When running on-premises servers (e.g., procurement from enterprise vendors like IBM or HP), a systems administrator or DevOps engineer must handle physical server installation, cooling, failed power supplies, bad RAM sticks, network cabling, hypervisor patches, and OS-level security updates. Managing 1,000 internal physical servers demands substantial staff overhead. In AWS, the entire physical layer is managed by AWS under the Shared Responsibility Model.

2. Pay-As-You-Go Economics

Physical servers involve non-refundable Capital Expenditures (CapEx). If hardware runs idle during weekends, overnight, or prolonged holiday shutdowns (e.g., Christmas or Diwali), you continue paying for depreciation, rack space, power, and cooling.

With EC2, computing resources fall strictly under an Operational Expenditure (OpEx) model. If workloads do not require execution over holidays or outside business hours, instances can be stopped or terminated, immediately pausing compute billing.

3. EC2 Instance Families and Workload Mapping

EC2 instances are optimized across distinct operational profiles depending on the hardware bottlenecks of the target application (CPU, memory, storage I/O, or floating-point compute).

Note: For foundational learning, testing, and standard CI/CD servers (like single-node Jenkins), the General Purpose family (t2.micro) satisfies all operational requirements within the AWS Free Tier.

4. Architectural Foundations: Regions vs. Availability Zones

AWS segregates its global physical infrastructure into two primary deployment abstractions: Regions and Availability Zones (AZs).

Regions

An AWS Region is a distinct geographic physical location (e.g., us-east-1 in North Virginia, eu-central-1 in Frankfurt, ap-south-1 in Mumbai).

Engineers select specific Regions based on two constraints:

1. Compliance and Data Sovereignty: European banking or healthcare systems often have legal mandates (such as GDPR) prohibiting customer records from leaving the European continent or country borders. An infrastructure engineer will explicitly provision resources within the eu-central-1 (Frankfurt) region.
2. Network Latency Minimization: Latency is defined as the total round-trip time required for an IP packet to traverse between a client and a target server. Deploying instances physically proximate to the end consumer minimizes packet round-trip time, preventing degraded user experiences.

Availability Zones (AZs)

An Availability Zone is one or more distinct, physical, enterprise-grade data centers located within a specified region, engineered with redundant power, networking, and flood protection.

• AZs inside a region are separated by miles to guard against local natural disasters, yet linked together by ultra-low-latency private fiber optics.
• They are abstracted using alphabetic characters (e.g., us-east-1a, us-east-1b).
• High Availability (HA) is achieved by distributing duplicate application instances across two or more AZs. If one data center suffers an isolated electrical failure or localized utility severed connection, traffic seamlessly shifts to surviving AZs without system downtime.

5. Free Tier Safeguards and Instance Provisioning Mechanics

Understanding the 750-Hour Free Tier Allocation

AWS grants new accounts 750 hours per month of compute usage for a t2.micro (or t3.micro where available) instance.

Because 744 hours is less than the 750-hour allowance, you can run exactly one t2.micro instance non-stop, 24/7 for a complete month at zero compute cost.

Warning: The 750-hour pool is shared across all running instances. If you launch two concurrent t2.micro instances:

Your monthly allocation will deplete in approximately 15.6 days ($\frac{750}{48} \approx 15.625$), after which standard on-demand hourly pricing will be billed to your account.

Step-by-Step Provisioning Choices

1. Amazon Machine Image (AMI): The software template (Operating System, pre-installed software, system architecture). Choose a clean distribution such as Ubuntu 22.04 LTS (64-bit x86) marked Free tier eligible.
2. Instance Type: Select t2.micro (1 vCPU, 1 GiB System RAM).
3. Key Pair: Asymmetric cryptographic token (RSA) used to authenticate SSH requests. Enter a label (e.g., AWS_login) and save the private .pem file to your local computer.
4. Storage Volume: Default root partition uses General Purpose SSD (gp2/gp3). Free Tier grants up to 30 GB of total EBS storage; the standard 8 GB allocation for an Ubuntu root partition is sufficient.

6. Secure Shell Access: Key-Pairs and the File Permission Trap

AWS disables password-based authentication by default on Linux AMIs to prevent automated brute-force attacks over port 22. Authentication relies on asymmetric cryptography (Public/Private Key Pair):

• Public Key: Retained on the provisioned EC2 instance within ~/.ssh/authorized_keys.
• Private Key (.pem): Downloaded to your local client machine during provisioning.

Real-World Debugging Workflow: The Open File Permission Trap

When initiating an SSH connection via terminal, modern OpenSSH clients evaluate the file system permissions of the private identity key:

# Attempt initial connection
ssh -i ~/Downloads/AWS_login.pem ubuntu@<PUBLIC_EC2_IP>

The SSH client frequently aborts the handshake with this fatal warning:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for 'AWS_login.pem' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "AWS_login.pem": bad permissions
ubuntu@<PUBLIC_EC2_IP>: Permission denied (publickey).

The Fix

The default permissions (644 or -rw-r--r--) allow other local system accounts to read your private key. You must restrict the file to read-only access for the file owner using chmod 600:

# Restrict permissions strictly to Owner (Read + Write), blocking Group and World
chmod 600 ~/Downloads/AWS_login.pem

# Re-execute the SSH command
ssh -i ~/Downloads/AWS_login.pem ubuntu@<PUBLIC_EC2_IP>

You are now successfully authenticated into the remote cloud environment as user ubuntu.

7. Hands-on Walkthrough: Deploying Jenkins on an Ubuntu EC2 Instance

Once authenticated into the remote host, update the system repository lists before installing new software packages:

# Refresh local APT package index
sudo apt update -y

Step 1: Install Java Runtime Environment (Prerequisite)

Jenkins is built on Java. It requires a compatible Java Development Kit (JDK) or Java Runtime Environment (JRE) to initialize its underlying JVM:

# Install OpenJDK 11 Headless / Full Development Kit
sudo apt install -y openjdk-11-jdk

# Verify installation and system path binding
java -version

Step 2: Install Jenkins Server

Fetch and import the official Jenkins repository GPG key, add the repository path to the local sources list, and execute the installation:

# Import the Jenkins Debian repository GPG keyring
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null

# Register the Jenkins stable distribution into APT sources
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

# Update package cache to detect Jenkins packages
sudo apt update -y

# Install Jenkins package daemon
sudo apt install -y jenkins

Step 3: Validate Daemon Operational State

Check that the systemd process manager has started the Jenkins service:

sudo systemctl status jenkins

The output must show Active: active (running):

● jenkins.service - Jenkins Continuous Integration Server
     Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-10-25 10:15:32 UTC; 1min ago
   Main PID: 14502 (java)
      Tasks: 42 (limit: 1143)
     Memory: 412.5M

8. Network Security: Debugging the Port 8080 Access Block

After verifying that the Jenkins daemon is active on the server, try to open the Jenkins web portal in a local browser:

http://<PUBLIC_EC2_IP>:8080

The browser will spin continuously and eventually display: ERR_CONNECTION_TIMED_OUT

Root Cause Analysis: The Default Security Group Firewall

AWS EC2 instances are protected by a stateful virtual firewall called a Security Group (SG).

• By default, Security Groups apply an explicit Deny All posture to inbound traffic unless a rule specifically allows it.
• During initial instance setup, only Port 22 (SSH) is open.
• Because Jenkins listens for plain HTTP traffic on Port 8080, inbound packets arriving from the Internet are dropped at the virtual perimeter of the AWS hypervisor before reaching your virtual server's kernel.

The Resolution: Modifying Inbound Rules

1. In the AWS Management Console, navigate to the EC2 Dashboard.
2. Select your running instance and click on the Security tab.
3. Click the hyperlinked Security Group ID (e.g., sg-01a23b4c5d6e7f890).
4. Click Edit inbound rules.
5. Select Add rule and configure the parameters:
   • Type: Custom TCP
   • Port range: 8080
   • Source: Anywhere-IPv4 (0.0.0.0/0)
6. Click Save rules.

Return to your browser and reload http://<PUBLIC_EC2_IP>:8080. The "Unlock Jenkins" installation portal will load immediately.

Retrieving the Initial Administrator Secret

To unlock Jenkins and set up the admin user, read the initialization token generated during package installation:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the 32-character hexadecimal alphanumeric string, paste it into the Jenkins web interface, and proceed with the initial plugin setup.

9. Complete End-to-End Execution Flow

This end-to-end architecture traces how administrative access and external client traffic reach your deployment:

Summary and Key Takeaways

• EC2 Represents Scalable Virtualization: Amazon Elastic Compute Cloud (EC2) provisions isolated, elastic virtual machines (instances) running on top of AWS-managed physical hypervisors.
• Maintenance Offloading: EC2 eliminates hardware acquisition delays, bare-metal server racking, hypervisor maintenance, and idle compute expenses through pay-as-you-go billing.
• Instance Types Match Workload Bottlenecks: Choose from General Purpose (balanced), Compute Optimized (CPU-heavy), Memory Optimized (high RAM), Storage Optimized (high disk IOPS), or Accelerated Computing (GPUs) depending on your application requirements.
• Regions Ensure Compliance and Speed: Regions provide geographic proximity to users to minimize latency and meet data sovereignty/compliance mandates (e.g., GDPR in Europe).
• Availability Zones Guard Against Outages: AZs are physically separate, redundant data centers within a single Region that allow you to design high-availability and fault-tolerant architectures.
• Free Tier Capacity Rules: The 750 free hours per month for t2.micro covers exactly one instance running 24/7. Running multiple instances concurrently burns through this shared pool proportionally faster.
• Private Key Hardening: AWS rejects insecure SSH key files. You must use chmod 600 <keyname>.pem to restrict local file permissions before OpenSSH will authenticate with an EC2 host.
• Security Groups Act as Stateful Firewalls: By default, EC2 instances drop all inbound traffic not explicitly permitted. To access web applications like Jenkins (port 8080), you must add an inbound rule allowing traffic from 0.0.0.0/0.