Day 14: Integrating Docker with GitLab CI/CD Pipelines

In modern DevOps, containerization is a foundational pillar. Almost every production-grade CI/CD pipeline is designed to package applications into lightweight, portable container images, push them to a secure registry, and deploy them to orchestrators like Kubernetes.

This guide explores how to integrate Docker with GitLab CI/CD. We will cover the core mechanics of containerization, the architecture of Docker in Docker (DinD), and how to build, tag, and push Docker images to the integrated GitLab Container Registry.

1. The Evolution of Application Packaging: Why Docker Matters

To understand why containerization is essential in CI/CD pipelines, consider how applications were traditionally run and shared.

Real-Life Analogy: The Java JAR File Problem

Imagine you build a Java-based application. Once compiled, you obtain an executable artifact: abc.jar. To run this application, you execute:

java -jar abc.jar

While this seems straightforward, running this JAR file successfully depends on several environmental factors:

• Java Runtime Environment (JRE): The host machine must have the exact or compatible version of Java installed.
• Port Availability: The application might require port 8080 to be open and free from conflicts.
• Execution Knowledge: The operator must know the exact execution commands and environment variables.
• Platform Discrepancies: An application built and tested on Linux might behave differently when run on Windows or macOS.

If you share only the abc.jar file with a colleague, they may encounter errors because their local machine lacks the correct Java version, has port conflicts, or runs an incompatible operating system.

The Docker Solution

Instead of sharing only the raw application artifact (the JAR file), Docker packages the application along with its entire runtime environment. This package includes:

1. The application artifact (abc.jar).
2. The required runtime (e.g., Java JRE).
3. System tools, libraries, and configurations.
4. Metadata specifying which ports to expose (e.g., port 8080) and the default execution command.

When your colleague receives this package, they run a single command to spin up an isolated environment called a Container. The application runs identically on any machine, eliminating the "works on my machine" problem.

2. Core Mental Model: Blueprints, Images, and Containers

To work effectively with Docker, you must understand its three core components:

 Dockerfile (Blueprint)
        |
        | docker build
        v
  Docker Image (Snapshot)
        |
        | docker run
        v
Docker Container (Running Instance)

• Dockerfile: A text configuration file containing step-by-step instructions on how to assemble the application package. It defines the base operating system, environment variables, files to copy, and commands to run.
• Docker Image: A read-only, static snapshot of the packaged application and its dependencies, built directly from the instructions in the Dockerfile.
• Docker Container: A runnable, isolated instance of a Docker Image. It is the actual "box" inside which your application executes.

3. The Architecture of a CI/CD Docker Pipeline Flow

In a automated DevOps workflow, the lifecycle of a containerized application follows a structured path from code commit to deployment:

4. Deep Dive: GitLab Container Registry vs. Alternatives

Once a Docker image is built inside a pipeline, it must be stored in a centralized repository known as a Container Registry.

Key Advantages of the GitLab Container Registry

1. Native Integration: Every project in GitLab has its own dedicated container registry out of the box.
2. Zero-Config Authentication: The pipeline does not require manual setup of external registry credentials; it uses built-in, short-lived environment tokens.
3. Security: Images are private by default, ensuring that only authorized users and deployment environments can pull them.

5. The Challenge of Running Docker inside Docker (DinD)

GitLab CI/CD jobs run inside Runners. When using the Docker Executor, the runner itself spins up a Docker container to execute your pipeline steps.

This creates a structural challenge: How do you execute Docker commands (like docker build or docker push) inside a job container that is already running inside Docker?

To solve this, we use a pattern called Docker in Docker (DinD). DinD allows a container to run a fully functional Docker daemon inside itself, enabling nested container operations.

6. The DinD Architecture: Job Container vs. Service Container

To make DinD work, GitLab CI/CD splits the responsibilities between two distinct containers running side-by-side within the runner: the Job Container and the Service Container.

1. Job Container (Docker CLI)

This container runs the primary image specified in your .gitlab-ci.yml (e.g., image: docker:24). It contains only the Docker Command Line Interface (CLI). It does not run the Docker engine itself. When you execute docker build, this container simply formats the command and sends it over the network.

2. Service Container (Docker Daemon)

This container runs as a background service (e.g., services: [docker:24-dind]). It hosts the Docker Daemon (dockerd), which is the core engine that builds images, manages layers, and communicates with registries.

Communication Protocols: TLS vs. Non-TLS

By default, the Docker daemon secures communication using Transport Layer Security (TLS). However, in a controlled CI/CD runner environment, setting up and sharing TLS certificates between the Job and Service containers adds complexity.

For simplicity, pipelines often disable TLS, switching communication to plain TCP:

• Port 2376: Used for secure, TLS-encrypted communication.
• Port 2375: Used for non-TLS, plaintext communication.

Note: To disable TLS in GitLab CI/CD, set the environment variable DOCKER_TLS_CERTDIR to an empty string (""). This instructs the daemon to accept plain TCP connections on port 2375.

7. GitLab Predefined Environment Variables for Container Registries

GitLab automatically injects several predefined environment variables into every pipeline run. These variables allow you to authenticate and interact with your project's container registry without hardcoding sensitive credentials.

8. Step-by-Step Implementation: Creating the Node.js Application and Dockerfile

Let's build and containerize a simple Node.js application.

Step 1: Create the Application Code

Create a directory named app and add a file named app.js:

// app/app.js
console.log("Hello Team! Day 14 Docker Integration with GitLab CI/CD is successful!");

Step 2: Create the Dockerfile

In the root directory of your project, create a file named Dockerfile (ensure the 'D' is capitalized and there is no file extension):

# Use an official lightweight Node.js runtime as the base image
FROM node:18

# Set the working directory inside the container
WORKDIR /app

# Copy the local app folder from the repository to the container's working directory
COPY app /app

# Define the default command to execute when the container starts
CMD ["node", "app.js"]

Explanation of Dockerfile Instructions:

• FROM node:18: Pulls a pre-configured Linux image that has Node.js version 18 and npm pre-installed.
• WORKDIR /app: Creates a directory named /app at the root of the container and sets it as the active working directory.
• COPY app /app: Copies the contents of your local app directory into the container's /app directory.
• CMD ["node", "app.js"]: Specifies the command that runs inside the container when it starts up.

9. Step-by-Step Implementation: Writing the .gitlab-ci.yml Pipeline Configuration

Create a .gitlab-ci.yml file in the root of your repository to automate the build and push process:

# Define the global Docker CLI image to run the pipeline steps
image: docker:24

# Spin up the Docker-in-Docker (DinD) service container to run the Docker Daemon
services:
  - docker:24-dind

# Configure global variables for DinD communication
variables:
  # Instruct the Docker CLI to connect to the Docker Daemon over plain TCP
  DOCKER_HOST: tcp://docker:2375
  # Disable TLS by leaving the certificate directory empty
  DOCKER_TLS_CERTDIR: ""

# Define the pipeline stages
stages:
  - build

# Define the build job
build_image:
  stage: build
  script:
    # 1. Print Docker system information for debugging purposes
    - docker info
    
    # 2. Authenticate the Docker CLI with the GitLab Container Registry
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    
    # 3. Build the Docker image and tag it as 'latest'
    - docker build -t "$CI_REGISTRY_IMAGE:latest" .
    
    # 4. Push the built image to the GitLab Container Registry
    - docker push "$CI_REGISTRY_IMAGE:latest"

Code Walkthrough:

1. image: docker:24: Loads the Docker CLI toolset into the job container.
2. services: - docker:24-dind: Starts the background Docker daemon container.
3. DOCKER_HOST: tcp://docker:2375: Tells the CLI container to send its commands to the hostname docker (the default alias for the service container) on port 2375.
4. DOCKER_TLS_CERTDIR: "": Disables TLS verification, allowing the CLI and Daemon to communicate over unencrypted TCP.
5. docker login: Uses the predefined GitLab environment variables to log in securely to the project's registry.
6. docker build -t ... .: Compiles the image using the current directory (.) as the build context and tags it with the project's registry path.

10. Verifying the Build and Exploring the GitLab Container Registry

Once you commit and push your code, GitLab CI/CD will trigger the pipeline.

git add .
git commit -m "feat: add Node.js app, Dockerfile, and GitLab CI/CD pipeline"
git push origin main

Verifying in the GitLab UI:

1. Navigate to your project on GitLab.
2. Go to Build > Pipelines to monitor the execution.
3. Once the job succeeds, go to Deploy > Container Registry in the left sidebar.
4. You will see your newly published image repository (e.g., registry.gitlab.com/username/day-14-docker-integration) containing an image tagged latest.

11. Pulling and Running the Container Locally

To verify that the image stored in the GitLab Container Registry works correctly, pull and run it on your local machine.

Step 1: Generate a Personal Access Token (PAT)

Because the GitLab Container Registry is private, your local Docker daemon must authenticate before pulling the image.

1. In GitLab, click on your profile picture in the top-left corner and select Preferences.
2. Select Access Tokens from the sidebar.
3. Click Add new token.
4. Name the token (e.g., local-docker-token).
5. Select the following scopes:
   • read_registry (allows pulling images)
   • write_registry (allows pushing images)
6. Click Create personal access token and copy the generated token immediately.

Step 2: Log In Locally

Open your local terminal and run:

docker login registry.gitlab.com

When prompted:

• Username: Enter your GitLab username.
• Password: Paste the Personal Access Token (PAT) you generated.

Step 3: Pull the Image

Copy the image path from the GitLab Container Registry UI and run:

docker pull registry.gitlab.com/<your-username>/day-14-docker-integration:latest

Step 4: Run the Container

Run the container in interactive, detached mode (-itd) and override the entrypoint with bash to inspect its contents:

docker run -itd --name test-node-app registry.gitlab.com/<your-username>/day-14-docker-integration:latest bash

Step 5: Verify Container Contents

Execute an interactive terminal session inside the running container:

docker exec -it test-node-app bash

Once inside the container, run:

# List files to verify app.js is present
ls

# View the contents of app.js
cat app.js

# Execute the application manually
node app.js

You should see the console output: Hello Team! Day 14 Docker Integration with GitLab CI/CD is successful!

To exit the container, type:

exit

12. Summary and Key Takeaways

• Docker's Core Value: Docker solves the "works on my machine" problem by packaging applications, runtimes, system tools, and configurations into a single portable image.
• GitLab Container Registry: Every GitLab project includes a built-in, secure container registry that integrates with your pipelines without requiring manual credential configuration.
• Docker in Docker (DinD): Because GitLab runners execute jobs inside containers, building Docker images requires a nested container architecture (DinD).
• Separation of Concerns in DinD: The Job Container runs the Docker CLI to send commands, while the Service Container runs the Docker Daemon (dockerd) to execute container operations.
• TLS Configuration: Setting DOCKER_TLS_CERTDIR: "" disables TLS verification inside the runner, allowing the CLI and Daemon to communicate over plain TCP on port 2375.
• Predefined Variables: GitLab provides automatic environment variables ($CI_REGISTRY_USER, $CI_REGISTRY_PASSWORD, and $CI_REGISTRY_IMAGE) to simplify registry authentication.
• Local Verification: Private GitLab images can be pulled and run locally by authenticating your local Docker CLI using a GitLab Personal Access Token (PAT) with read_registry scopes.