SonarQube Tutorial for DevOps | Introduction to DevSecOps

This chapter explores the integration of security into modern continuous integration and continuous delivery (CI/CD) pipelines from a DevOps perspective. It covers foundational DevSecOps concepts, distinguishes between key security testing methodologies, and provides a step-by-step guide to setting up an end-to-end production-grade pipeline using SonarQube, Jenkins, Maven, and Docker.

1. The Paradigm Shift: Why DevSecOps & Shifting Left

In traditional software development lifecycles (SDLC), security checks were treated as a final, isolated phase before release. This approach introduces significant bottlenecks, high remediation costs, and operational friction.

The Problems with Late Security Checks

• Release Failures and Rework: Discovering a critical vulnerability right before deployment forces the development team to halt the release, trace the bug back to its origin, rewrite code, and restart the entire build and test cycle.
• Slow, Inconsistent, and Unauditable Manual Reviews: Relying on manual security audits is slow, prone to human error, and lacks an automated audit trail.
• Accumulation of Hidden Risks: Modern applications rely heavily on third-party libraries. Without automated scanning, transitive dependencies can introduce silent vulnerabilities that accumulate over time.

The Solution: Shift Left

"Shift Left" is the practice of moving security testing as early as possible in the SDLC.

• Requirement Gathering: If the team decides to deploy containerized workloads, the DevOps engineer must immediately plan how to secure both the static container images and the running containers.
• Development: Developers receive real-time security feedback directly in their Integrated Development Environments (IDEs) via plugins.
• Build and Test: Automated security gates block vulnerable code from merging into main branches.

2. Core Mental Model: Functional vs. Security Testing

To design an effective pipeline, you must distinguish between testing whether an application works (functional testing) and whether it is secure (security testing).

Code Coverage vs. Security

Code coverage is a functional testing metric. It measures the percentage of code executed during automated tests:

Note: High code coverage does not mean the code is secure. An application can have $100\%$ code coverage while containing severe SQL injection vulnerabilities or hard-coded secrets. Functional tests only verify that the code behaves as expected under specified inputs.

3. Key DevSecOps Methodologies: SAST, DAST, SCA, and SBOM

DevSecOps relies on several automated testing methodologies. These are industry practices rather than specific tools; various commercial and open-source utilities implement them.

Key Distinctions

• SAST vs. DAST: SAST scans the code "at rest" without executing it. DAST tests the active application "in motion" by simulating external attacks against its running endpoints.
• SCA vs. SBOM: SCA actively scans dependencies to find vulnerabilities and license violations. An SBOM is a static, machine-readable inventory (e.g., in SPDX or CycloneDX format) of those dependencies. If a new Common Vulnerabilities and Exposures (CVE) entry is published after release, security teams can scan the stored SBOM to identify affected production workloads without rerunning active scans on the application.

4. Deep Dive into SonarQube: Code Quality, SAST, and Code Coverage

SonarQube is an automatic code review tool that detects bugs, vulnerabilities, and code smells in your project. It acts as a central quality gate in the CI/CD pipeline.

1. Static Application Security Testing (SAST)

SonarQube classifies security findings into three categories:

• Vulnerabilities: Actual security flaws in the code that an attacker can exploit (e.g., SQL injection). These require immediate remediation.
• Security Hotspots: Code patterns that are security-sensitive and require human review to determine if they pose a risk (e.g., using a weak encryption algorithm or a self-signed certificate in a private environment).
• Security-Sensitive Bugs: Functional bugs that directly impact the application's security posture.

2. Code Quality

SonarQube measures code maintainability and readability by identifying:

• Code Smells: Poorly structured code that is not necessarily broken but makes future modifications difficult and error-prone.
• Duplication: Repeated blocks of code that should be refactored into reusable functions.
• Technical Debt: The estimated effort (typically measured in minutes or days) required to fix code smells and maintainability issues.

3. Code Coverage Integration

SonarQube does not generate code coverage reports itself. Instead, it relies on external coverage tools to run during the build phase, generate a report, and import that report during the analysis phase.

5. End-to-End Production DevSecOps Workflow (Java + Maven)

The following workflow illustrates how security checks integrate into a production-grade CI/CD pipeline for a Java application built with Maven.

Pipeline Optimization: The Redundant Build Pitfall

A common mistake in pipeline design is running redundant build commands. For example, running mvn verify followed by mvn package or mvn jar:jar in later stages is unnecessary.

The verify phase in Maven's default lifecycle runs after the package phase:

Because verify has already compiled, tested, and packaged the application into an artifact (e.g., a .jar file), the pipeline should reuse that existing artifact for the Docker build stage rather than rebuilding it. Eliminating redundant compilation steps reduces pipeline execution times.

6. SonarQube Architecture: Server vs. Scanner

SonarQube uses a client-server architecture to decouple code analysis from reporting and storage.

1. SonarQube Server

The server runs as a central service and consists of three main components:

• Web Server: Serves the user interface and exposes the HTTP APIs used by scanners to upload reports.
• Compute Engine: Processes the uploaded analysis reports, calculates metrics, checks code against quality profiles, and evaluates Quality Gate conditions.
• Database: Stores project metadata, historical analysis runs, user configurations, and quality profiles.

Note: While SonarQube includes an embedded H2 database for testing, it is not supported for production. Production deployments require an external database like PostgreSQL, Microsoft SQL Server, or Oracle.

2. SonarQube Scanner

The scanner is a stateless client utility that runs locally on build agents, developer machines, or within CI pipelines. It reads the local source code, collects test coverage reports, packages the data into an analysis payload, and sends it to the SonarQube server over HTTP. Scanners are available as standalone binaries or integrated plugins for build tools like Maven and Gradle.

7. Step-by-Step Lab: Provisioning and Configuring SonarQube with PostgreSQL

This lab configures a production-grade SonarQube instance on an AWS EC2 instance running Ubuntu 24.04, backed by a local PostgreSQL database.

Step 7.1: System Configuration & Kernel Tweaks

SonarQube uses an embedded Elasticsearch instance for search functionality. Elasticsearch requires specific system limits to run stably.

Connect to your SonarQube EC2 instance and set the host name and time zone:

sudo hostnamectl set-hostname sonarqube
sudo timedatectl set-timezone UTC

Apply kernel parameter adjustments by appending limits to /etc/sysctl.conf:

sudo bash -c 'cat <<EOF >> /etc/sysctl.conf
vm.max_map_count=524288
fs.file-max=131072
EOF'

# Apply the changes immediately
sudo sysctl -p

Configure file limits in /etc/security/limits.conf:

sudo bash -c 'cat <<EOF >> /etc/security/limits.conf
sonar   soft    nofile          131072
sonar   hard    nofile          131072
sonar   soft    nproc           8192
sonar   hard    nproc           8192
EOF'

Step 7.2: Install Java Runtime Environment (JRE)

SonarQube is a Java-based application. Because this server only hosts the SonarQube application and does not compile code, we install the JRE rather than the full JDK.

sudo apt update
sudo apt install -y openjdk-17-jre

Step 7.3: Install and Configure PostgreSQL

Install PostgreSQL and its dependencies:

sudo apt install -y postgresql postgresql-contrib
sudo systemctl enable postgresql
sudo systemctl start postgresql

Log into the PostgreSQL interactive terminal as the default postgres user:

sudo -i -u postgres psql

Run the following SQL commands to provision a dedicated database and user for SonarQube:

-- Create a dedicated database user
CREATE ROLE sonar WITH LOGIN PASSWORD 'sonar';

-- Create the database with the sonar user as owner
CREATE DATABASE sonarqube OWNER sonar;

-- Grant all privileges on the database to the sonar user
GRANT ALL PRIVILEGES ON DATABASE sonarqube TO sonar;

-- Exit the PostgreSQL terminal
\q

Verify the database configuration:

psql -h localhost -U sonar -d sonarqube -W
# Enter password 'sonar' when prompted, then exit using \q

Step 7.4: Download and Install SonarQube

Create a dedicated system user for SonarQube:

sudo useradd -r -m -s /bin/bash sonar

Download and extract the SonarQube Community Build:

cd /tmp
sudo apt install -y unzip wget
wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-10.6.0.92116.zip -O sonarqube.zip

# Unzip and move to the optional directory
unzip sonarqube.zip
sudo mv sonarqube-10.6.0.92116 /opt/sonarqube-current

# Grant ownership of the directory to the sonar user
sudo chown -R sonar:sonar /opt/sonarqube-current

Step 7.5: Configure SonarQube Properties

Edit the SonarQube configuration file to point to the PostgreSQL database:

sudo -u sonar vi /opt/sonarqube-current/conf/sonar.properties

Uncomment and modify the following properties:

sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:postgresql://localhost:5432/sonarqube

Step 7.6: Create a Systemd Service File

To manage SonarQube as a system service, create a unit file at /etc/systemd/system/sonarqube.service:

[Unit]
Description=SonarQube service
After=syslog.target network.target postgresql.service

[Service]
Type=forking
ExecStart=/opt/sonarqube-current/bin/linux-x86-64/sonar.sh start
ExecStop=/opt/sonarqube-current/bin/linux-x86-64/sonar.sh stop
User=sonar
Group=sonar
Restart=always
LimitNOFILE=131072
LimitNPROC=8192

[Install]
WantedBy=multi-user.target

Reload systemd, enable, and start the service:

sudo systemctl daemon-reload
sudo systemctl enable sonarqube
sudo systemctl start sonarqube

Verify that SonarQube is running:

sudo systemctl status sonarqube

Access the SonarQube web UI by navigating to http://<SONAR_EC2_PUBLIC_IP>:9000 in your browser. Log in using the default credentials (admin / admin) and update the password when prompted.

8. Step-by-Step Lab: Provisioning Jenkins, Maven, and Docker

This lab configures the CI build server on a separate EC2 instance running Ubuntu 24.04.

Step 8.1: System Configuration & JDK Installation

Connect to your Jenkins EC2 instance and set the host name and time zone:

sudo hostnamectl set-hostname jenkins
sudo timedatectl set-timezone UTC

Install the Java Development Kit (JDK). We install the JDK here because this server compiles and packages Java applications.

sudo apt update
sudo apt install -y openjdk-17-jdk

Step 8.2: Install Jenkins

Add the Jenkins repository keys and install the service:

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian-stable binary/" | sudo tee \
  /etc/apt/sources.list.get.jenkins.list > /dev/null

sudo apt update
sudo apt install -y jenkins

sudo systemctl enable jenkins
sudo systemctl start jenkins

Retrieve the initial administrator password:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Navigate to http://<JENKINS_EC2_PUBLIC_IP>:8080, paste the password, select Install Suggested Plugins, and create your admin user.

Step 8.3: Install Maven and Docker

Install Maven to manage Java builds:

sudo apt install -y maven
mvn -version

Install Docker to build and run container images:

# Add Docker's official GPG key:
sudo apt update
sudo apt install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io

Step 8.4: Configure Permissions

To allow Jenkins to run Docker commands without root privileges, add the jenkins user to the docker group:

sudo usermod -aG docker jenkins
sudo usermod -aG docker ubuntu

# Restart Jenkins to apply group membership changes
sudo systemctl restart jenkins

9. Hands-on: Preparing the Vulnerable Maven Project & Git Integration

This section covers setting up a private Git repository, creating a Maven project, and introducing code smells and security vulnerabilities to test our pipeline.

Step 9.1: Create a Private GitHub Repository

1. Go to GitHub and create a new private repository named my-app-private.
2. Generate a Personal Access Token (PAT) with repo scope by navigating to Settings > Developer Settings > Personal Access Tokens > Tokens (classic). Save this token securely.

Step 9.2: Generate the Maven Project

On the Jenkins server (acting as our development environment), generate a new Maven project:

cd /tmp
mvn archetype:generate \
  -DgroupId=com.cloudwithvjosh \
  -DartifactId=my-app \
  -DarchetypeArtifactId=maven-archetype-quickstart \
  -DarchetypeVersion=1.5 \
  -DinteractiveMode=false

cd my-app

Step 9.3: Configure JaCoCo in pom.xml

Edit the pom.xml file to add the JaCoCo plugin inside the <plugins> block. This plugin records code coverage data during test execution.

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.11</version>
    <executions>
        <execution>
            <id>prepare-agent</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
        <execution>
            <id>report</id>
            <phase>test</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Step 9.4: Introduce Vulnerabilities and Code Smells

Replace the contents of /tmp/my-app/src/main/java/com/cloudwithvjosh/App.java with the following code. This code contains a hardcoded password (a security hotspot) and an unused variable (a code smell).

package com.cloudwithvjosh;

public class App {
    // Code Smell: Unused private field
    private static final String UNUSED_FIELD = "I am not used anywhere";

    public static void main(String[] args) {
        System.out.println("Hello World!");
        
        // Security Hotspot: Hardcoded credentials
        String dbPassword = "SuperSecretAdminPassword123!";
        connectToDatabase(dbPassword);
    }

    private static void connectToDatabase(String password) {
        System.out.println("Connecting to database with password length: " + password.length());
    }
}

Step 9.5: Push Code to the Private Repository

Initialize Git, commit the files, and push them to your private GitHub repository:

git init
git add .
git commit -m "Initial commit with vulnerable code and JaCoCo plugin"
git branch -M main

# Add the remote repository using your GitHub PAT for authentication
git remote add origin https://<YOUR_GITHUB_USERNAME>:<YOUR_GITHUB_PAT>@github.com/<YOUR_GITHUB_USERNAME>/my-app-private.git
git push -u origin main

10. Hands-on: Configuring SonarQube Projects, Users, and Quality Gates

Before running the pipeline, configure the target project, service user, and Quality Gate in SonarQube.

Step 10.1: Create the Project

1. Log into SonarQube (http://<SONAR_IP>:9000).
2. Click Create Project and select Manually.
3. Set the Project Key and Display Name to my-app.
4. For the New Code definition, select Previous Version (the default setting) and click Create Project.

Step 10.2: Create a Service User and Token

1. Navigate to Administration > Security > Users.
2. Click Create User. Set the login name to jenkins-sonar-user and fill in the details.
3. Once created, click the Tokens icon next to the user.
4. Generate a token named jenkins-token and copy it.

Step 10.3: Assign Administrator Permissions

1. Navigate to Administration > Security > Global Permissions.
2. Ensure the jenkins-sonar-user has permissions to administer quality gates and perform analysis.

Step 10.4: Create a Custom Quality Gate

By default, SonarQube applies the "Sonar Way" quality gate. We will create a custom gate to enforce a strict code coverage threshold.

1. Navigate to Quality Gates and click Create.
2. Name it my-app-gate.
3. Click Add Condition and select Overall Code.
4. Choose Coverage as the metric, set the operator to is less than, and enter 80 as the value.
5. Under the Projects section of the quality gate, associate it with the my-app project.

11. Hands-on: Building the Jenkins Pipeline & Analyzing the Quality Gate Failure

Now, configure a Jenkins job to run the analysis and observe how the Quality Gate behaves.

Step 11.1: Add Credentials in Jenkins

1. Go to Manage Jenkins > Credentials > System > Global credentials (unrestricted).
2. Click Add Credentials.
3. Select Username with password for GitHub. Use your GitHub username and the PAT as the password. Set the ID to github-creds.
4. Click Add Credentials again. Select Secret text for SonarQube. Paste the SonarQube user token into the Secret field and set the ID to sonar-token.

Step 11.2: Create a Jenkins Freestyle Job

1. Click New Item, name it my-app-sonar-job, select Freestyle project, and click OK.
2. Under Source Code Management, select Git:
   • Repository URL: https://github.com/<YOUR_GITHUB_USERNAME>/my-app-private.git
   • Credentials: Select github-creds.
   • Branch Specifier: */main
3. Under Build Environment, check Use secret text(s) or file(s):
   • Add a Secret Text variable.
   • Set the Variable name to SONAR_TOKEN and select the sonar-token credential.

Step 11.3: Add the Build Script

Add an Execute shell build step with the following script. Replace <SONAR_IP> with your SonarQube server's private or public IP address.

#!/bin/bash
echo "=== Workspace Contents ==="
ls -la

# Define SonarQube Server IP
SONAR_IP="<YOUR_SONAR_IP>"

echo "=== Running Maven Build, Tests, and SonarQube Scan ==="
# We run clean and verify (which packages the app and runs tests).
# Then we invoke the sonar-maven-plugin to upload results.
mvn clean verify sonar:sonar \
  -Dsonar.projectKey=my-app \
  -Dsonar.host.url=http://$SONAR_IP:9000 \
  -Dsonar.token=$SONAR_TOKEN \
  -Dsonar.qualitygate.wait=true

echo "=== Building Docker Image ==="
# Create a Dockerfile on the fly
cat <<EOF > Dockerfile
FROM openjdk-17-slim
WORKDIR /app
COPY target/my-app-1.0-SNAPSHOT.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]
EOF

# Build the image
docker build -t my-java-app:latest .

echo "=== Running Container ==="
# Remove existing container if it exists
docker rm -f my-running-app || true

# Run the new container
docker run -d --name my-running-app my-java-app:latest

Step 11.4: Run the First Build (Expected Failure)

Click Build Now to run the job.

The build fails during the Maven phase. The console output shows:

Because we set -Dsonar.qualitygate.wait=true, the scanner polls the server and blocks the pipeline if the Quality Gate conditions are not met. This prevents untested or vulnerable code from being packaged into a Docker image.

Step 11.5: Fix the Quality Gate and Rerun

In a production environment, developers would resolve this by writing unit tests to increase code coverage. For this lab, we will adjust the Quality Gate to let the pipeline proceed:

1. Go to the SonarQube UI, navigate to Quality Gates > my-app-gate.
2. Delete the coverage condition.
3. Go back to Jenkins and click Build Now.

The build now completes successfully. Jenkins compiles the code, passes the Quality Gate, builds the Docker image, and starts the container.

12. Summary and Key Takeaways

• Shift Left Security: Integrating security checks early in the SDLC reduces remediation costs and prevents late-stage release failures.
• Functional vs. Security Metrics: High code coverage does not guarantee secure code. Functional testing verifies behavior, while security testing (SAST, DAST, SCA) identifies vulnerabilities.
• SAST vs. DAST: SAST analyzes source code at rest during the build phase. DAST tests the running application over network interfaces in staging environments.
• SCA and SBOM: SCA scans dependencies for active vulnerabilities. SBOMs provide a machine-readable inventory of components to simplify post-release security tracking.
• Pipeline Optimization: Avoid running redundant compilation commands (like mvn package after mvn verify) to keep pipeline runtimes low.
• Decoupled Architecture: SonarQube separates the analysis engine (Scanner) from the reporting and storage engine (Server and Database).
• Quality Gates as Circuit Breakers: Configuring pipelines to wait for Quality Gate results (-Dsonar.qualitygate.wait=true) prevents non-compliant artifacts from reaching production.