Mastering GitLab CI/CD: Reuse Common Job Definitions by Reference (YAML Anchors/References)

1. Why Do We Need This? (The Problem of Pipeline Code Duplication)

When building complex continuous integration and continuous deployment (CI/CD) pipelines in GitLab, developers frequently encounter a common maintenance bottleneck: repetition. As projects grow to dozens or hundreds of jobs, you often find yourself copying and pasting identical blocks of code—such as standard build scripts, environment variable declarations, execution conditions (rules), setup commands (before_script), and cleanup routines (after_script)—across multiple job definitions within your .gitlab-ci.yml file.

The Maintenance Nightmare

• High Technical Debt: If a shared build step or environment variable changes (e.g., upgrading a package manager or changing a cluster endpoint), you must manually update every single job definition.
• Human Error: Missing even one job during manual updates leads to inconsistent pipeline behaviors, hard-to-debug failures, and broken builds.
• Bloated Configuration: Large YAML files become difficult to read, review, and maintain over time.

To solve this, GitLab CI/CD leverages underlying YAML features to allow developers to define common, reusable job blocks and reference them wherever needed.

2. Core Mental Model & The YAML Hidden Anchor Concept

The reference mechanism used in GitLab CI/CD is not actually a feature documented under native GitLab CI/CD reference pages; rather, it is a powerful native utility of the underlying YAML parser specification.

Real-Life Analogy: The Master Blueprint and Photocopies

Analogy: Imagine an architect designing a housing development with fifty identical kitchens. Instead of drawing the exact same kitchen layout from scratch fifty times over in the master blueprint book, the architect draws one Master Kitchen Blueprint labeled .common-kitchen and places a reference note in each house: "Build kitchen matching Master Kitchen Blueprint." If the builder needs to update the countertop material, they change it once in the master blueprint, and all houses automatically inherit the update.

In GitLab CI/CD, we create hidden template jobs (prefixed with a dot . so GitLab ignores them as standalone executable jobs) and use YAML anchor/reference syntax to pull their contents into active jobs.

3. Naive vs. Better Approach: Code Duplication vs. Referenced Templates

Naive Approach (Repeated Code)

In a naive setup, every single stage repeats the exact same configuration blocks:

stages:
  - compile
  - test

job_one:
  stage: compile
  script:
    - echo "Running default script"
    - echo "Installing dependencies"
    - echo "Building application"

job_two:
  stage: test
  script:
    - echo "Running default script"
    - echo "Installing dependencies"
    - echo "Running unit tests"

Issue: If the installation steps change, you must update both job_one and job_two.

Better Approach (Using YAML References)

By creating a common job template and referencing its script section, we eliminate duplication entirely:

# Hidden Common Job Template (Prefixed with a dot so GitLab ignores it as a real job)
.common_job_template:
  script:
    - echo "Running default script"
    - echo "Installing some packages"

stages:
  - compile

compile_job:
  stage: compile
  script:
    -,--------------------------------------------------
    # Reference the script section from the template
    - !reference [.common_job_template, script]
    - echo "Running job-specific compile steps"

4. Step-by-Step Evolution of GitLab YAML References

Let's walk through how to progressively scale this technique from simple script reuse to variables and execution rules.

Step A: Referencing Entire Script Blocks

You can pull an entire sequence of commands from a template directly into a job's script array.

.common_job_template:
  script:
    - echo "Running default script"
    - echo "Installing dependencies..."

stages:
  - compile

my_compile_job:
  stage: compile
  script:
    - !reference [.common_job_template, script]

Step B: Combining Shared References with Job-Specific Commands

Often, you want the shared setup commands to run first, followed immediately by unique commands tailored to that specific job. The !reference tag lets you mix and match inline.

stages:
  - test

.common_job_template:
  script:
    - echo "Running default script"
    - echo "Installing dependencies..."

test_one:
  stage: test
  script:
    # 1. Pull shared baseline script commands
    - !reference [.common_job_template, script]
    # 2. Append job-specific commands
    - echo "Running specific test suite A"
    - echo "Executing validation checks"

Execution Flow Walkthrough

5. Referencing Environment Variables Across Jobs

Beyond scripts, you can also template and reference configuration blocks such as variables. This ensures that common configuration values, API URLs, or environment tags are maintained centrally.

.common_job_template:
  variables:
    ENVIRONMENT: "staging"
    APP_VERSION: "1.0.4"
  script:
    - echo "Base setup complete"

test_two:
  stage: test
  variables:
    # Reference variables from the template job
    - !reference [.common_job_template, variables]
  script:
    - !reference [.common_job_template, script]
    - echo "Verifying environment variables..."
    - echo "Target Environment is $ENVIRONMENT with Version $APP_VERSION"

6. Referencing Pipeline Execution Rules (rules)

You can even template complex conditional logic (rules) to control when jobs are included or skipped, ensuring consistent CI/CD policies across your entire repository.

.common_job_template:
  rules:
    # Prevent automatic execution on standard branch git push commits
    - if: '$CI_PIPELINE_SOURCE == "push"'
      when: never
    # Allow manual execution via web UI pipelines
    - if: '$CI_PIPELINE_SOURCE == "web"'
      when: always

test_three:
  stage: test
  rules:
    - !reference [.common_job_template, rules]
  script:
    - echo "Executing test_three job triggered via Web UI pipeline"

Real-World Debugging & Pipeline Behavior

• Branch Push vs. Web Trigger: When a developer pushes a normal commit to the repository branch, the rule matching if: '$CI_PIPELINE_SOURCE == "push"' evaluates to when: never, causing test_three to be completely omitted from the active pipeline view.
• Manual Web Trigger: When running a pipeline manually through the GitLab Web UI (CI_PIPELINE_SOURCE == "web"), the second rule matches, successfully injecting test_three into the running pipeline graph.

7. Comprehensive Feature Matrix: What Can Be Referenced?

Almost any standard GitLab CI/CD job keyword block can be stored in a common template and pulled via YAML references.

8. Summary and Key Takeaways

• YAML Native Utility: GitLab CI/CD references (!reference) rely on native YAML parsing capabilities rather than proprietary GitLab platform documentation.
• Eliminate Duplication: Centralize repeated scripts, environment variables, rules, and configurations into shared master blocks to reduce technical debt.
• The Dot Prefix Convention (.): Always prefix template jobs with a dot (.common_job_template) so GitLab treats them as hidden anchors rather than executing them as standalone jobs in every pipeline stage.
• Flexible Composition: You can combine shared references with custom, job-specific inline commands to maintain both standardization and flexibility.
• Granular Targeting: References can target specific subsections of a template job independently, such as pulling just the script array or just the variables dictionary.
• Consistent Governance: Template rules and artifacts across projects to enforce team-wide CI/CD standards and security guardrails.