Day 5: Build, Push & Deploy a Dockerized Flask App with Jenkins | Project-Based Approach

This chapter covers the complete implementation of a Continuous Integration and Continuous Deployment (CI/CD) workflow using Jenkins to containerize, publish, and deploy a Python Flask application. It explores the architectural differences between Docker in Docker (DinD) and Docker Outside of Docker (DooD), demonstrates the mechanics of mounting UNIX domain sockets, examines secure credential binding, and provides an automated deployment flow targeting the host Docker engine.

1. Architectural Foundations: Docker in Docker (DinD) vs. Docker Outside of Docker (DooD)

To build Open Container Initiative (OCI) compliant images within a CI/CD pipeline, the execution environment must have access to container-building tooling (such as Docker CLI/Daemon, Buildah, or Kaniko). When Jenkins runs as a container, it lacks direct access to container runtime mechanics by default.

There are two primary patterns for enabling Docker capabilities within a containerized Jenkins controller or agent:

The Role of /var/run/docker.sock

In a standard Linux environment, the Docker architecture decouples the client (docker CLI) from the server daemon (dockerd). They communicate using one of two mechanisms:

1. Network sockets via TCP/IP (e.g., tcp://0.0.0.0:2375 or TLS-secured on 2376).
2. UNIX domain sockets (IPC - Inter-Process Communication) located at /var/run/docker.sock.

A UNIX domain socket functions entirely within the operating system kernel, bypassing the networking subsystem, routing tables, and the TCP/IP stack. It provides a lightweight, high-throughput communication channel between local processes.

When mounting /var/run/docker.sock from the host system into the Jenkins container, the container-internal Docker CLI sends standard REST API requests across this shared socket directly to the host’s Docker daemon. As a result, containers created by the Jenkins job run as sibling containers on the host machine rather than nested containers inside Jenkins.

2. Configuring Docker Outside of Docker (DooD) on Jenkins

To configure DooD, recreate the Jenkins container with the host's UNIX domain socket mounted as a bind mount alongside the persistent jenkins_home named volume.

Step 1: Re-instantiating Jenkins with the Socket Mount

First, stop and remove any running Jenkins container:

docker stop jenkins
docker rm jenkins

Next, deploy the Jenkins container, mounting both the configuration directory and the Docker socket:

docker run -d \
  --name jenkins \
  --restart unless-stopped \
  -p 8080:8080 \
  -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e TZ="America/New_York" \
  jenkins/jenkins:lts

• -v jenkins_home:/var/jenkins_home: Preserves jobs, plugins, configuration history, and credentials across container lifecycles.
• -v /var/run/docker.sock:/var/run/docker.sock: Exposes the host's Docker daemon control socket to the container's filesystem.

Step 2: Socket Permissions and User Management

By default, /var/run/docker.sock is owned by root and assigned to the host's docker group with read/write permissions (rw-rw---- or mode 660). Because the official Jenkins image runs under an unprivileged non-root user (jenkins, UID 1000), executing Docker commands initially results in a permission denied error.

To grant the jenkins user access to the socket file for local testing:

# Access the container as root
docker exec -u 0 -it jenkins bash

# Relax permissions on the mounted socket
chmod 666 /var/run/docker.sock

# Exit the container
exit

Note: Assigning chmod 666 grants read/write permissions to all users on the host. In hardened production environments, avoid chmod 666. Instead, create a group inside the container matching the exact GID of the host's docker group and assign the jenkins user to that group.

Step 3: Installing the Docker CLI Inside Jenkins

The official jenkins/jenkins base image does not include the Docker client binary. Install the Docker CLI inside the container:

# Exec into the running container as root
docker exec -u 0 -it jenkins bash

# Update packages and install Docker CLI dependencies
apt-get update && apt-get install -y ca-certificates curl gnupg

# Configure Docker repository keyring
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg

# Add Docker repository to APT sources
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install the docker-ce-cli package exclusively (omit dockerd)
apt-get update && apt-get install -y docker-ce-cli

# Exit root shell
exit

Step 4: Verification of Client-Daemon Binding

Log into the container as the unprivileged jenkins user to verify that the CLI communicates with the host engine:

docker exec -it jenkins bash
docker version

The output confirms that the Docker CLI running within the container connects to the host's Docker Engine:

Client: Docker Engine - Community
 Version:           28.4.0
 API version:       1.47
 Go version:        go1.23.6
 Git commit:        d8da76e
 Built:             Wed Feb 19 18:06:55 2025
 OS/Arch:           linux/arm64

Server: Docker Desktop 4.37.1 (178610)
 Engine:
  Version:          28.4.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.23.6
  Git commit:       465ec87
  Built:            Wed Feb 19 18:06:55 2025
  OS/Arch:          linux/arm64

3. The Target Application: Python Flask Microservice

The microservice consists of a lightweight web application that exposes a single HTTP GET route on TCP port 5000.

Application Source Code (app.py)

from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello():
    return (
        "<h1>Welcome to Cloud With Varun</h1>"
        "<p>Built with Flask | Shipped by Jenkins | Running in Docker</p>"
    )

if __name__ == '__main__':
    # Listen on all available network interfaces
    app.run(host='0.0.0.0', port=5000)

Dependency Specification (requirements.txt)

flask==3.0.0

Containerization Strategy (Dockerfile)

# Stage: Base runtime layer using minimal Debian-slim Python
FROM python:3.11-slim

# Define working directory inside the container
WORKDIR /app

# Optimize layer caching: Copy and install requirements separately
COPY requirements.txt .

# Install dependencies without retaining pip's download cache
RUN pip install --no-cache-dir -r requirements.txt

# Copy source code into the active working directory
COPY app.py .

# Informational metadata exposing port 5000
EXPOSE 5000

# Specify runtime execution entrypoint
CMD ["python", "app.py"]

Layer Optimization Principles

1. Targeted Dependency Copying: Copying requirements.txt prior to copying application source code prevents application changes from invalidating the cached dependency layer (RUN pip install ...).
2. Cache Elimination: The --no-cache-dir flag prevents pip from writing wheel and package tarballs to ~/.cache/pip, reducing the final image footprint.
3. Explicit Bindings: The application binds to 0.0.0.0 within app.py instead of the loopback interface (127.0.0.1), ensuring incoming traffic routed through Docker port mapping reaches the process.

4. Secure Credential Architecture: GitHub & Docker Hub

Jenkins provides a centralized, encrypted storage system for API tokens, SSH keys, and passwords. When configuring credentials, their scope determines which parts of Jenkins can access them:

• Global Scope: Credentials are accessible across all items, including Freestyle projects, Multi-branch pipelines, and agents.
• System Scope: Restricts credentials strictly to the Jenkins controller infrastructure (e.g., node provisioning, mail servers), shielding them from build jobs.

1. GitHub Authentication via Personal Access Token (PAT)

To pull private repositories over HTTPS:

1. Navigate to GitHub -> Settings -> Developer Settings -> Personal Access Tokens (Classic).
2. Generate a token with the repo scope.
3. In Jenkins, go to Manage Jenkins -> Credentials -> System -> Global credentials -> Add Credentials.
4. Configure the credential:
   • Kind: Username with password
   • Username: A descriptive identifier (e.g., git-pat-user)
   • Password: The generated GitHub PAT
   • ID: github-pat

2. Docker Hub Authentication

To publish container images to Docker Hub:

1. Under Global credentials, select Add Credentials.
2. Configure the credential:
   • Kind: Username with password
   • Username: Your Docker Hub registry username (e.g., cloudwithvarun)
   • Password: The Docker Hub personal access token or account password
   • ID: dockerhub-creds

3. Safe Password Injection via Stdin

Injecting credentials using plain command-line arguments (such as docker login -u $USER -p$PASS) exposes secrets in process listings (ps -ef) and system telemetry logs. Instead, use standard input injection:

echo "$DOCKER_HUB_PWD" \vert  docker login -u "$DOCKER_HUB_USER" --password-stdin

The Jenkins Credentials Binding Plugin intercepts standard output and error descriptors during job execution, replacing matched secret values with asterisks (****) to prevent accidental credential leakage in the build logs.

5. Artifact Tagging Strategies and Traceability

Tracking container artifacts back to their originating source code, commits, and build runs is a fundamental continuous delivery requirement.

Dynamic Tagging Implementation

In this workflow, the built image is tagged twice:

1. Pinned Build Number ($BUILD_NUMBER): Creates a distinct, immutable release tag correlating directly to the Jenkins execution log.
2. Floating Latest Tag (latest): References the most recent successful build, allowing local environments and consumers to pull updates without manual version tracking.

6. End-to-End Build, Push, and Deploy Automation

The complete orchestration logic runs as a single shell execution step within the Jenkins workspace.

Complete Shell Execution Script

#!/bin/bash
set -e # Terminate immediately if any command returns a non-zero exit status

# ---------------------------------------------------------
# Step 1: Authenticate to Docker Hub using injected secrets
# ---------------------------------------------------------
echo "Authenticating with container registry..."
echo "$DOCKER_HUB_PWD" \vert  docker login -u "$DOCKER_HUB_USER" --password-stdin

# ---------------------------------------------------------
# Step 2: Establish naming conventions & tagging metadata
# ---------------------------------------------------------
REGISTRY_USER="${DOCKER_HUB_USER}"
APP_NAME="cwvj-flask"
IMAGE_NAME="${REGISTRY_USER}/${APP_NAME}"
TAG="${BUILD_NUMBER}"

echo "Target Image: ${IMAGE_NAME}:${TAG}"

# ---------------------------------------------------------
# Step 3: Build container image with dual tags
# ---------------------------------------------------------
echo "Initiating image build..."
docker build \
  -t "${IMAGE_NAME}:${TAG}" \
  -t "${IMAGE_NAME}:latest" \
  .

# ---------------------------------------------------------
# Step 4: Push artifacts to Docker Hub
# ---------------------------------------------------------
echo "Pushing artifacts to remote registry..."
docker push "${IMAGE_NAME}:${TAG}"
docker push "${IMAGE_NAME}:latest"

# ---------------------------------------------------------
# Step 5: Deploy the updated container onto the Docker Host
# ---------------------------------------------------------
CONTAINER_NAME="flask-app-prod"

echo "Deploying container: ${CONTAINER_NAME}..."

# Remove existing container instance if present (prevents name collision)
if [ "$(docker ps -aq -f name=^/${CONTAINER_NAME}$)" ]; then
    echo "Stopping and removing existing deployment..."
    docker rm -f "${CONTAINER_NAME}"
fi

# Run the updated container instance on host port 5000
docker run -d \
  --name "${CONTAINER_NAME}" \
  -p 5000:5000 \
  --restart unless-stopped \
  "${IMAGE_NAME}:${TAG}"

# ---------------------------------------------------------
# Step 6: Smoke Test Validation
# ---------------------------------------------------------
echo "Executing smoke validation..."
sleep 2

# Query local deployment
if curl -s -f http://localhost:5000 > /dev/null; then
    echo "Smoke test passed: Application is reachable and serving traffic."
else
    echo "Smoke test failed: Unable to reach application on port 5000."
    exit 1
fi

# ---------------------------------------------------------
# Step 7: Workspace Cleanup
# ---------------------------------------------------------
echo "Cleaning ephemeral Jenkins scratch space..."
rm -rf ${WORKSPACE}/*

7. Deployment Orchestration and Container Lifecycles

Because Jenkins connects directly to the host's Docker socket, running docker run inside the build step spawns the application container directly on the host operating system.

Preventing Container Name Collisions

Docker enforces unique container names. Attempting to deploy a container with a name already assigned to an existing instance produces an error:

docker: Error response from daemon: Conflict. The container name "/flask-app-prod" is already in use by container "...".

To achieve idempotent deployments, the build script removes any existing container sharing the target name before starting the new instance:

docker rm -f flask-app-prod || true

• -f (--force): Uses SIGKILL to stop running containers before removal.
• || true: Prevents shell script aborts when no matching container exists on the initial run.

Workspace Hygiene

Every Jenkins project workspace acts as scratch storage on the disk at: /var/jenkins_home/workspace/<job-name>/

Leaving build artifacts, downloaded dependencies, and Git histories in this directory across builds consumes disk space over time. Removing workspace files using rm -rf ${WORKSPACE}/* at the end of the build script preserves disk capacity on the host.

8. Verification, Debugging, and Persistence Mechanics

Verifying Image Layers and Remote Registry Storage

Once pushed to Docker Hub, both tags share an identical SHA-256 content digest because their underlying layers are identical:

# Output from Docker Hub layer verification
cloudwithvarun/cwvj-flask:1      Digest: sha256:87a9e32a6bc8ef...
cloudwithvarun/cwvj-flask:latest Digest: sha256:87a9e32a6bc8ef...

The underlying filesystem layers are only pushed to the remote registry once. The second tag simply creates an additional pointer to the existing manifest, minimizing network transfer and storage usage.

Verifying Application Deployment

Confirm the container is running and serving traffic on the host machine:

# Inspect container state
docker ps --filter "name=flask-app-prod"

# Verify port connectivity via HTTP request
curl -i http://localhost:5000

Expected HTTP response headers:

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 104
Server: Werkzeug/3.0.0 Python/3.11.8
Date: Mon, 10 Mar 2025 12:00:00 GMT

<h1>Welcome to Cloud With Varun</h1><p>Built with Flask | Shipped by Jenkins | Running in Docker</p>

Credential Persistence on the Controller

All configured credentials persist in /var/jenkins_home/credentials.xml. Within this file, passwords and tokens are encrypted using a system-generated master key stored in /var/jenkins_home/secrets/master.key.

Because /var/jenkins_home is mounted to a named Docker volume (jenkins_home), all encrypted credentials, build histories, and pipeline configurations survive container restarts, upgrades, and host reboots.

9. Troubleshooting and Common Failure Modes

1. Permission Denied on /var/run/docker.sock

• Symptom: Jenkins build log fails with:
  CODE

  Copy

  Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock
  

• Root Cause: The unprivileged jenkins user (UID 1000) inside the container does not have read/write access to the host's socket file.
• Resolution: Execute chmod 666 /var/run/docker.sock on the host, or align the container user's secondary group with the host's Docker group ID:
  BASH

  Copy

  docker exec -u 0 jenkins usermod -aG $(stat -c '%g' /var/run/docker.sock) jenkins
  

2. Docker Login Exposure Warning

• Symptom: Pipeline logs show the warning:
  CODE

  Copy

  WARNING! Using --password via the CLI is insecure. Use --password-stdin.
  

• Root Cause: Passing passwords directly via CLI flags leaves them visible in process tables (ps).
• Resolution: Pipe the password variable through standard input:
  BASH

  Copy

  echo "$PASSWORD_VARIABLE" \vert  docker login -u "$USER_VARIABLE" --password-stdin
  

3. Port Conflicts During Deployment

• Symptom: The deployment step fails with:
  CODE

  Copy

  docker: Error response from daemon: driver failed programming external connectivity on endpoint flask-app-prod: Bind for 0.0.0.0:5000 failed: port is already allocated.
  

• Root Cause: Another service or an orphaned container is actively bound to host port 5000.
• Resolution: Identify the conflicting process with lsof -i :5000 or netstat -tulnp | grep 5000, stop the conflicting process, or rebind the application container to an unused host port (e.g., -p 5001:5000).

10. Summary and Key Takeaways

• Docker Outside of Docker (DooD) avoids the storage driver conflicts and --privileged container overhead of Docker in Docker (DinD) by mounting /var/run/docker.sock from the host system into the Jenkins container.
• UNIX Domain Sockets allow inter-process communication directly through the operating system kernel, bypassing the overhead and security risks of exposing the Docker daemon over an unencrypted TCP port.
• Sibling Containers: Because the Jenkins container shares the host's Docker daemon, containers spawned by build jobs run as sibling containers on the host rather than nested containers inside Jenkins.
• Layer Optimization: Structuring the Dockerfile to copy and install requirements.txt before copying application source code leverages layer caching to significantly reduce rebuild times.
• Standard Input Secret Injection: Using echo "$SECRET" | docker login --password-stdin protects credentials from exposure in system process tables and telemetry tools.
• Credential Masking: The Jenkins Credentials Binding Plugin automatically injects secrets into build environment variables and masks their values with asterisks (****) in console outputs.
• Build Traceability: Tagging images with the dynamic $BUILD_NUMBER variable establishes a clear audit trail between running production images and the specific Jenkins jobs that built them.
• Idempotent Deployments: Checking for and removing existing containers (docker rm -f) prevents deployment failures caused by name collisions when deploying updated application containers.
• Workspace Cleanup: Removing scratch directories (rm -rf ${WORKSPACE}/*) at the end of the build cycle prevents disk exhaustion on continuous integration nodes.