GitLab CI/CD: Explore Artifacts (Files/Directories) Transfer in Pipeline

This comprehensive guide explores Job Artifacts in GitLab CI/CD, detailing how files and directories generated during one job can be securely stored, transferred, and reused across subsequent jobs in your pipeline.

1. Why Do We Need Job Artifacts? (The Problem Statement)

In GitLab CI/CD, every single job runs inside its own isolated, separate environment (conceptually like a fresh, dedicated container or virtual machine).

When a job executes—such as compiling a Java project, building assets, or generating test reports—any files or folders created during that run remain strictly contained within that specific worker machine's temporary workspace.

The Status Quo & Limitation

• Isolated Execution: Job A runs on Machine 1, generating build classes or text files. Job B runs on Machine 2.
• No Shared Local Storage by Default: Job B cannot automatically see or access files created in Job A.
• Redundant Work: Without a transfer mechanism, downstream jobs (like testing or packaging) would be forced to re-compile or regenerate files from scratch, wasting compute time and resources.

2. Core Mental Model: The Postal Storage Locker

Real-Life Analogy

Imagine an assembly line in a factory. Worker 1 works at Desk A building engine components. When finished, Worker 1 cannot simply throw the parts across the room to Worker 2 at Desk B. Instead, Worker 1 places the completed parts into a central storage locker (GitLab's Artifact Storage) with a label. Worker 2, needing those parts for the next assembly stage, walks over to the locker, retrieves the package, and continues the work.

3. How Job Artifacts Work

GitLab provides a built-in feature called Job Artifacts that bridges this isolation gap.

1. Generation: A job creates files or folders during its execution script (e.g., example.txt).
2. Archiving: You configure the artifacts keyword in your .gitlab-ci.yml file, specifying the file path.
3. Upload: GitLab automatically captures those files and uploads them to a secure, centralized GitLab storage location.
4. Consumption: Downstream jobs declare dependencies (via needs or explicit configurations) and automatically download those artifacts into their local workspace to continue processing.

4. Step-by-Step Configuration: Implementing Artifacts in .gitlab-ci.yml

Let's look at how to implement this in a pipeline configuration. We will create a compile job that generates a file (example.txt), saves it as an artifact, and then passes it to downstream test jobs.

Naive Approach vs. Artifact-Enabled Approach

stages:
  - build
  - test

compile_job:
  stage: build
  script:
    - echo "Hello World" > example.txt
    - ls -la
  artifacts:
    paths:
      - example.txt
    expire_in: 30 days

test_job_1:
  stage: test
  needs:
    - compile_job
  script:
    - ls -la
    - cat example.txt

5. Anatomy of the artifacts Keyword

The artifacts block supports several vital sub-configurations:

6. Complete Flow Walkthrough & Execution Behavior

Let's examine how different jobs interact with artifacts based on dependency configuration (needs):

Key Execution Insight

When you use the needs keyword to establish a job dependency, GitLab automatically handles artifact downloading under the hood.

• If a source job has stored artifacts, the dependent job downloads them.
• If the source job did not store artifacts (or if no dependency is declared, like test_job_2), the files will be missing, causing file-reading commands (like cat) to fail.

7. Inspecting Artifacts in the GitLab UI

Once your pipeline finishes running, GitLab provides robust web UI inspection tools:

1. Job Logs: View the upload confirmation in the job console output:

   TEXT

   Copy

   Uploading artifacts...
   example.txt: Found 1 matching artifact files and directories
   Uploading artifacts to coordinator... ok
   

2. Download Button: On the job details page, a side panel provides a direct Download button that packages all job artifacts into a downloadable .zip archive.

3. Browse Artifacts: You can click Browse to navigate the internal directory structure of the stored artifacts directly inside the browser and inspect file contents (e.g., viewing Hello World inside example.txt).

8. Summary and Key Takeaways

• Environment Isolation: Every GitLab CI/CD job runs in a separate, isolated environment; local files are not shared between jobs by default.
• Job Artifacts Feature: Artifacts provide a seamless mechanism to capture files and folders generated during a job and store them in secure GitLab storage.
• The artifacts: paths Directive: Used in .gitlab-ci.yml to point to specific files or directories that need to be preserved.
• Automatic Expiration (expire_in): Helps manage storage by automatically purging artifacts after a defined timeframe (defaulting to 30 days in many configurations).
• Dual-Purpose Dependency (needs): Declaring job dependencies not only controls execution order but also automatically triggers artifact downloading from upstream jobs.
• Missing File Pitfall: Downstream jobs that lack dependencies or point to jobs without stored artifacts will fail if they attempt to read non-existent files.
• UI Verification: GitLab allows users to browse, inspect, and download artifact archives directly from the pipeline web interface.
• Next Steps: In upcoming tutorials, we will explore transferring CI/CD Variables across jobs as artifacts.