Deployment & CI/CD Automation: Simplifying Workflows with Makefiles

This chapter covers the transition from raw shell scripting to structured automation using Makefiles in DevOps pipelines. While shell scripts are highly effective for sequential tasks (such as provisioning an AWS EC2 instance), managing complex, multi-step Docker and application workflows can quickly become unwieldy. Here, we explore how to integrate shell commands into declarative, clean, and reusable Makefiles to simplify local development, deployment, and CI/CD automation.

1. Why Do We Need Makefiles in DevOps?

In a typical DevOps workflow, engineers frequently run long, repetitive commands to build, tag, run, and clean up containerized applications.

The Problem: Command Fatigue and Complexity

Consider a standard Docker-based workflow:

• Building an image: docker build -t my-app:latest .
• Tagging an image: docker tag my-app:latest my-registry.com/my-app:v1
• Running a multi-container stack: docker-compose up -d
• Cleaning up dangling resources: docker system prune -f

Remembering and typing these commands repeatedly is error-prone and inefficient. While you could write individual shell scripts for each action (e.g., build.sh, deploy.sh), managing multiple script files adds unnecessary clutter to your repository.

The Solution: Makefiles as a Command Wrapper

A Makefile is a special configuration file used by the make build automation tool. It allows you to define short, memorable aliases (called targets) for complex shell commands. Instead of typing a long Docker command, you simply run:

make build
make up
make clean

2. Core Mental Model & Analogy

Real-Life Analogy: The Universal Remote Control

Imagine your home entertainment system has a TV, a soundbar, a streaming device, and a gaming console. To watch a movie, you would normally have to:

1. Pick up the TV remote and turn it on.
2. Pick up the soundbar remote and set the input to "Optical."
3. Pick up the console remote and boot up the media app.

This is equivalent to running individual, manual shell commands.

A Makefile acts like a Universal Remote Control programmed with "Macro" buttons:

• Pressing the "Movie Mode" button (equivalent to make up) automatically triggers the sequential steps to turn on the TV, switch inputs, and launch the app behind the scenes.
• Pressing the "Power Off" button (equivalent to make clean) shuts down all devices simultaneously.

3. Anatomy of a Makefile

A Makefile consists of variables and rules. Each rule defines a target, its dependencies, and the system commands to execute.

Syntax Structure

target: dependencies
<TAB>system_command

Warning: Makefiles are extremely strict about indentation. You must use a physical Tab character (\t) for command lines, not spaces. Using spaces will result in a Makefile: missing separator syntax error.

Key Components

• Variables: Used to store reusable strings, commands, or dynamic values (e.g., detecting the host Operating System).
• Targets: The label or alias you call from the terminal (e.g., build, run, clean).
• Commands: The actual shell commands executed when the target is called.

4. Step-by-Step Evolution: Writing a DevOps Makefile

Let's build a production-ready Makefile for a Two-Tier Flask Application that utilizes Docker and Docker Compose.

Step 1: Defining Variables

We define variables for our tools and dynamically detect the host operating system using shell execution:

# Define the Docker Compose command variable
DOCKER_COMPOSE := docker-compose

# Dynamically detect the host Operating System
OS := $(shell uname)

Step 2: Creating Targets for the Application Lifecycle

Now, we map our common development and deployment workflows to clean targets:

# Target to build the application and print OS info
build:
	@echo "Running build process on $(OS)..."
	$(DOCKER_COMPOSE) build

# Target to spin up the application in detached mode
up:
	@echo "Starting application containers..."
	$(DOCKER_COMPOSE) up -d

# Target to stop and tear down the application containers
down:
	@echo "Stopping application containers..."
	$(DOCKER_COMPOSE) down

# Target to clean up unused Docker resources
clean:
	@echo "Cleaning up unused Docker resources..."
	docker system prune -f

5. Setting Up the Environment

To test our Makefile, we clone a sample Two-Tier Flask Application repository and prepare our environment.

1. Clone the Repository

git clone https://github.com/example/two-tier-flask-app.git
cd two-tier-flask-app

2. Remove Existing Makefiles (If Any)

To practice writing it from scratch:

rm -f Makefile

3. Create and Edit the New Makefile

Open your terminal editor (such as vim or nano):

vim Makefile

Paste the Makefile configuration designed in Section 4, ensuring that all command lines are indented with a physical Tab key. Save and exit (:wq in Vim).

6. Real-World Troubleshooting & Debugging

During deployment, you will frequently encounter system-level errors. Below are the exact issues faced during the live demonstration and how to resolve them.

Issue 1: make: command not found

• Cause: The make utility is not installed by default on clean Linux distributions (like Ubuntu/Debian).
• Resolution: Install it using the system package manager:
  BASH

  Copy

  sudo apt update
  sudo apt install make -y
  

Issue 2: docker or docker-compose Command Not Found

• Cause: The host system does not have the Docker engine or Docker Compose binary installed.
• Resolution: Update your package index and install Docker and Docker Compose:
  BASH

  Copy

  sudo apt-get update
  sudo apt-get install docker.io docker-compose -y
  

Issue 3: Docker Socket Permission Denied

• Cause: By default, the Docker daemon binds to a Unix socket owned by the root user. Non-root users cannot access it, resulting in the following error:
  TEXT

  Copy

  Got permission denied while trying to connect to the Docker daemon socket at /var/run/docker.sock
  

• Resolution: Change the ownership of the Docker socket to the current logged-in user:
  BASH

  Copy

  sudo chown $USER /var/run/docker.sock
  

  Verify access by running:
  BASH

  Copy

  docker ps
  

7. Complete Workflow Execution Walkthrough

Once your environment is configured and permissions are resolved, you can execute your entire deployment pipeline using simple make commands.

Execution Flow

Running the Commands

1. Build the Application Stack:

   BASH

   Copy

   make build
   

   Output:

   TEXT

   Copy

   Running build process on Linux...
   Building web...
   Step 1/5 : FROM python:3.9-slim
   ...
   Successfully built...
   

2. Deploy the Containers:

   BASH

   Copy

   make up
   

   Output:

   TEXT

   Copy

   Starting application containers...
   Creating network "two-tier-app_default" with the default driver
   Creating db ... done
   Creating web ... done
   

3. Tear Down the Stack:

   BASH

   Copy

   make down
   

4. Clean System Garbage:

   BASH

   Copy

   make clean
   

8. Summary and Key Takeaways

• Makefiles Abstract Complexity: They act as a clean, unified interface over complex shell commands, making local development and CI/CD pipelines highly readable.
• Strict Tab Indentation: Always use a physical Tab character for commands under a target; spaces will break the Makefile parser.
• Dynamic Variables: Use $(shell <command>) to dynamically gather environment properties (like operating systems or Git commit hashes) to make your automation context-aware.
• Prerequisites Matter: Always ensure that system utilities like make, docker, and docker-compose are installed and updated before running automation scripts.
• Socket Permissions: When automating Docker tasks on new Linux instances, remember to grant socket permissions to your user using sudo chown $USER /var/run/docker.sock to avoid execution failures.