Day-16: Kubernetes Integration in GitLab CI/CD

1. Why Do We Need Kubernetes Integration in GitLab CI/CD?

When building modern cloud-native applications, code must eventually be packaged into container images and deployed to a container orchestrator like Kubernetes. Doing this manually introduces critical bottlenecks:

• Manual Overhead: Manual kubectl deployments are slow, tedious, and prone to human error.
• Automation Gap: Without an integrated CI/CD pipeline, developers have to jump between environments, build images locally, and manually apply manifests, defeating the purpose of continuous delivery.
• Authentication Hurdles: By default, connecting an external CI/CD tool to a secured Kubernetes cluster throws authentication and authorization errors unless proper access credentials and configuration files are supplied.

Note: Integrating Kubernetes directly into your GitLab CI/CD pipeline ensures that every successful build and test phase automatically triggers a secure, repeatable deployment to your target cluster.

2. Core Mental Model & Deployment Architecture

To understand how GitLab communicates with Kubernetes, we must trace the end-to-end flow of an automated deployment.

GitLab CI/CD Pipeline
   |
   | 1. Builds Docker Image & Pushes
   v
Docker Registry (e.g., DockerHub / ECR)
   |
   | 2. Updates Kubernetes Manifest (Deployment / Service)
   v
AWS EKS Cluster (Kubernetes API Server)
   |
   | 3. Pulls image and runs container pods
   v
Worker Nodes Running Application (Nginx)

Real-Life Analogy

Think of the GitLab pipeline as a construction foreman who needs to deliver newly manufactured prefabricated rooms (Docker images) into a secure, gated apartment complex (the Kubernetes cluster). The foreman cannot just walk in; they need a validated badge (AWS Security Credentials) and a master blueprint directory (the kubeconfig file / eksctl authentication token) to present to the security gatekeeper (the Kubernetes API server) before workers are allowed to hoist the rooms into place.

3. Prerequisites & Required Tools

Before executing pipeline-based deployments, you must provision and configure several core infrastructure components:

4. Setting Up the Connector Virtual Machine (AWS EKS)

To bootstrap our Kubernetes infrastructure, we provision a dedicated administrative virtual machine on AWS EC2.

Step-by-Step Provisioning & Tool Installation

1. Launch EC2 Instance:

   • AMI: Ubuntu 24.04 LTS
   • Instance Type: t3.medium
   • Storage: 15 GB EBS Volume
   • Security Group: Allow SSH (Port 22) using a PEM key pair (.pem).

2. Connect via MobaXterm: Use SSH to log into the connector VM as the ubuntu user:

   BASH

   Copy

   ssh -i "your-key.pem" ubuntu@<PUBLIC_IP>
   

3. Install Core Utilities & AWS CLI:

   BASH

   Copy

   sudo apt update
   sudo apt install -y unzip curl
   
   # Download and install AWS CLI v2
   curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
   unzip awscliv2.zip
   sudo ./aws/install
   aws --version
   

4. Install kubectl:

   BASH

   Copy

   curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
   chmod +x kubectl
   sudo mv kubectl /usr/local/bin/
   kubectl version --client
   

5. Install eksctl:

   BASH

   Copy

   ARCH=amd64
   PLATFORM=$(uname)_$ARCH
   curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
   tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
   sudo mv /tmp/eksctl /usr/local/bin
   eksctl version
   

5. Configuring AWS IAM & Creating the EKS Cluster

To prevent executing administrative commands as the root user, we create a dedicated IAM user with scoped administrator permissions.

Creating the IAM User

1. Navigate to the AWS IAM Console -> Users -> Create User.
2. Name the user eksuser and grant programmatic console access.
3. Attach the permission policy: AdministratorAccess (or equivalent high-level EKS/EC2 permissions).
4. Generate an Access Key ID and Secret Access Key.

Authenticating the Connector VM

Run the configuration command on your connector VM and supply your credentials:

aws configure

• AWS Access Key ID: <paste_access_key>
• AWS Secret Access Key: <paste_secret_key>
• Default region name: ap-south-1
• Default output format: (Leave blank)

Creating the EKS Cluster

Deploy a demo cluster utilizing eksctl with a single t3a.medium worker node:

eksctl create cluster \
  --name demo-cluster \
  --region ap-south-1 \
  --node-type t3a.medium \
  --nodes 1

Note: Cluster provisioning typically takes 10–15 minutes. Once complete, verify connectivity from the connector VM:

BASH

Copy

kubectl get nodes

6. Manual Kubernetes Manifest Deployment & Validation

Before automating deployments through GitLab CI/CD, validate that your Kubernetes manifests deploy and function correctly.

1. Create the Deployment Manifest (deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: demo-1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

2. Create the Service Manifest (service.yaml)

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: demo-1
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30007

3. Deploy and Test Manually

# Create the target namespace
kubectl create ns demo-1

# Apply all manifests in the current directory
kubectl apply -f . -n demo-1

# Verify running pods and services
kubectl get all -n demo-1

Production Debugging Reality (Security Group Rules): If you attempt to access http://<Worker-Node-IP>:30007 from your browser and it fails to load, inspect your EC2 Worker Node's security group. Ensure you have added inbound rules permitting traffic on Port 80 and the Kubernetes NodePort range (30000-32767) from all IP locations (0.0.0.0/0).

Clean up the manual deployment namespace before proceeding to CI/CD integration:

kubectl delete ns demo-1

7. Connecting GitLab CI/CD to AWS EKS (Authentication Strategy)

To allow GitLab pipelines to communicate with EKS without hardcoding static credentials or long-lived authentication keys, we configure secure CI/CD Environment Variables.

Required GitLab CI/CD Variables

Navigate to your GitLab Project -> Settings -> CI/CD -> Variables and add the following secret variables (ensuring Masked and Protected settings are properly checked where applicable):

8. Writing the GitLab CI/CD Deployment Pipeline (.gitlab-ci.yml)

Instead of manually passing static kubeconfig files (which can cause file size or formatting errors in pipeline environments), we dynamically generate the kubeconfig at runtime using AWS CLI authentication tokens.

Consolidated Pipeline Implementation

Create .gitlab-ci.yml in your repository root:

stages:
  - deploy

deploy_to_k8s:
  stage: deploy
  image: 
    name: amazon/aws-cli:latest
    entrypoint: [""]
  script:
    - echo "Installing kubectl..."
    - curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
    - chmod +x kubectl
    - mv kubectl /usr/local/bin/
    
    - echo "Verifying kubectl and AWS CLI..."
    - kubectl version --client
    - aws --version
    
    - echo "Configuring kubeconfig for EKS cluster..."
    - aws eks update-kubeconfig --region $AWS_DEFAULT_REGION --name $AWS_CLUSTER_NAME
    
    - echo "Verifying cluster connectivity..."
    - kubectl get nodes
    
    - echo "Applying Kubernetes deployment manifests..."
    - kubectl apply -f ds-svc.yaml
    
    - echo "Verifying deployed pods and services..."
    - kubectl get pods -o wide
    - kubectl get svc
  only:
    - main

Crucial Pipeline Detail (entrypoint: [""]): By default, the official amazon/aws-cli Docker image defines a default entrypoint command. Overriding entrypoint: [""] instructs GitLab to clear this default entrypoint, allowing GitLab to successfully execute standard shell commands (sh or bash) inside the runner container.

9. Real-World Debugging Pitfalls & Solutions

When integrating Kubernetes with GitLab pipelines, engineers frequently run into common errors. Here is how to diagnose and resolve them:

10. Complete Flow Walkthrough: From Commit to Cluster

11. Summary and Key Takeaways

• Automated Deployments: Integrating Kubernetes into your GitLab CI/CD pipeline removes manual bottlenecks, ensuring fast, repeatable, and error-free application releases.
• Dynamic Authentication: Instead of storing static, long-lived kubeconfig credentials (which risk expiration or leakage), generate configurations dynamically at runtime using aws eks update-kubeconfig.
• Image Entrypoint Override: When using specialized cloud CLI images like amazon/aws-cli, always configure entrypoint: ["'"] so GitLab can execute shell commands correctly.
• Targeted Manifest Application: Avoid wildcard applies (kubectl apply -f .) in production pipelines to prevent non-manifest repository files from crashing the deployment stage.
• Security Group Management: Always ensure worker node security groups allow inbound traffic on your application's designated NodePort range (30000-32767) to test services externally.
• Cost Optimization: Always clean up temporary cloud test infrastructure when finished by executing eksctl delete cluster --name <cluster-name> --region <region> to prevent unexpected cloud billing charges.