Design CD Jobs in GitLab Pipelines: Common Deployment Jobs for All Environments

This comprehensive guide covers the architectural design and strategic implementation of continuous deployment (CD) jobs in GitLab CI/CD pipelines. It focuses on setting up a centralized, reusable deployment mechanism across multiple non-production and production environments using parent-child pipelines and secure runner agents.

1. Why Do We Need a Centralized Deployment Strategy?

In a standard enterprise application lifecycle, code moves through multiple distinct deployment environments before reaching production. As the number of target environments scales, writing isolated, repetitive deployment scripts for every single job leads to massive code duplication and maintenance nightmares.

Target Deployment Environments

• Development (Dev): Rapid integration environment for developer testing.
• QA (Quality Assurance): Stable environment for automated and manual testing.
• Staging (Stage): Pre-production environment mirroring production constraints.
• Production (Prod): Live customer-facing environment.

2. Core Mental Model: The Factory Assembly Line Analogy

Real-Life Analogy: Think of an automobile manufacturing plant. You do not build a completely new set of assembly robots and tools for every single car model that rolls off the line. Instead, you build a standardized, centralized assembly station. When a specific car chassis arrives, you simply pass unique configuration parameters—such as paint color, tire size, and interior trim—into the exact same standardized workflow.

Applying this to GitLab CI/CD:

• The deployment process remains identical across all environments (stop app, download artifact, replace files, restart app).
• Only the parameters (target IP, target directory, environment name, credentials) change depending on whether you are deploying to Dev or Production.

3. Naive Approach vs. Centralized Modular Approach

The Naive Approach (Code Duplication)

Writing a dedicated .gitlab-ci.yml script inside every application project containing the full shell script for Dev, QA, Staging, and Production deployments:

• Disadvantage 1: High duplication of deployment logic across repositories.
• Disadvantage 2: Updating deployment logic requires modifying dozens of microservice repositories.
• Disadvantage 3: Prone to configuration drift and human error.

The Better Approach: Centralized Child Pipelines

Instead of duplicating scripts, we decouple deployment logic into a separate dedicated GitLab project (e.g., environment-deployment-project). The main application pipeline triggers this external pipeline as a Child Pipeline (or Multi-Project Pipeline), passing necessary environmental variables.

4. Step-by-Step Architecture of Multi-Project Child Pipelines

The architecture splits responsibilities between the Application Project and the Deployment Project:

Flow of Execution

1. Build Phase: The application builds its artifact (e.g., a Spring Boot .jar snapshot or release) and publishes it to the GitLab Artifact Repository.
2. Trigger Phase: The application pipeline invokes the deployment project via GitLab's multi-project trigger mechanism, supplying environment-specific parameters.
3. Execution Phase: The centralized deployment pipeline executes the common job using a dedicated runner agent.

5. Pipeline Agents and Network Topology in AWS (EC2 & Subnets)

To deploy securely onto backend EC2 instances living inside private or isolated networks, we place Pipeline Agents inside strategic public subnets with proper routing and SSH security group access.

Security & Connectivity Setup

• Network Isolation: Non-production and production agents are isolated to prevent accidental cross-environment deployments.
• SSH Key Authentication: Pipeline agents securely connect to target backend EC2 instances using private SSH keys configured via GitLab CI/CD secure variables.
• Parameter Injection: Dynamic parameters like TARGET_IP and TARGET_DIRECTORY are passed from the parent pipeline to inform the agent exactly which backend machine to target.

6. Step-by-Step Deployment Job Workflow

When the common deployment job triggers on the pipeline agent, it executes a standardized procedural script transferred directly to the target machine.

7. Standalone Pipeline Flexibility and Use Cases

By isolating the deployment logic into a separate repository, this architecture unlocks a powerful production capability: Standalone Manual Deployments.

Why Standalone Pipelines Matter

• Emergency Hotfixes & Rollbacks: If an artifact was built weeks ago and needs to be redeployed or pushed to a specific stage immediately, operators do not need to rerun the full heavy application build pipeline.
• Decoupled Release Cadence: Release engineers can manage infrastructure deployments independently of developer code commits.

8. Summary Comparison: Traditional vs. Centralized Deployment Architecture

9. Summary and Key Takeaways

• Eliminate Duplication: Avoid writing repetitive deployment scripts across individual application repositories by leveraging centralized common jobs.
• Multi-Project Pipelines: Use GitLab's child/multi-project pipeline feature to decouple application building from environment deployment.
• Parameterize Workflows: Keep deployment scripts generic while passing environment-specific variables (TARGET_IP, TARGET_DIRECTORY, ENVIRONMENT_NAME) at runtime.
• Secure Network Agents: Isolate pipeline runner agents in dedicated subnets with controlled SSH connectivity to backend EC2 instances.
• Stand-Alone Flexibility: Enable manual, on-demand deployments of historical artifacts without needing to trigger a full application source code rebuild.
• Standardized Execution Steps: Enforce a consistent deployment lifecycle across all environments: Stop App -> Transfer Artifact & Script -> Replace Files -> Start App.