Trivy EP10 | Scan Local File Systems with Trivy 🔍 | Detect Secrets & Vulnerabilities (trivy fs)

1. Introduction: Moving Beyond Docker Images to Local File System Scans

Up to this point in our DevSecOps journey, our focus has primarily been on scanning built Docker images and container artifacts. While container scanning is critical, it happens after the artifact has already been created and packaged.

However, security best practices dictate shifting left—catching vulnerabilities and security misconfigurations directly within our local workspace, source code repositories, and file systems before any container image or CI/CD pipeline run ever takes place.

• The Shift-Left Paradigm: Scanning code and configuration files directly on a local developer machine (trivy fs).
• Pre-Artifact Protection: Intercepting hardcoded secrets, misconfigurations, and software vulnerabilities before the code reaches GitHub or CI/CD pipelines.

2. Core Mental Model & Real-Life Analogy

Real-Life Analogy: The Secret Diary vs. Public Post

Think about a personal secret, such as the story of Ram and Sham discussed in the lecture. Ram tells Sham: "I ate two mangoes today; please don't share it with anyone." Sham, acting as a trusted confidant, keeps that information strictly private.

In software engineering and cloud infrastructure, a secret is any piece of sensitive data meant to be kept strictly confidential and never exposed to the public world. Just as you would not publish your private diary entries on a public billboard, you must never expose authentication credentials in source code. When secrets are hardcoded and pushed to public repositories, it is equivalent to broadcasting your private diary to the entire internet.

3. What is a "Secret" in Cloud & IT Engineering?

A secret is any sensitive piece of data that requires authentication and authorizes access to cloud infrastructure, databases, APIs, or source control management systems.

Common examples of secrets include:

• API Keys: Access tokens used by external services to authenticate programmatic requests.
• Access Tokens: OAuth or bearer tokens granting session access.
• Passwords: Hardcoded database credentials or administrative passwords.
• Private Keys: Cryptographic SSH keys (id_rsa) or GPG signing keys.
• JSON Web Tokens (JWTs): Signed tokens used for authorization and authentication claims.

Why Do Secret Leaks Happen?

According to industry analysis and real-world post-mortems, secret leaks typically happen due to three primary developer workflow anti-patterns:

1. Hardcoding for Speed: Developers hardcode tokens directly into source files to save time during local testing and bypass complex configuration dependencies.
2. Accidental File Commits: Sensitive files (such as .env or local key files) are accidentally added to Git staging and committed.
3. Blind Copying of Configuration Files: Developers copy boilerplates or template configuration files containing placeholder credentials and fail to sanitize them.

4. The Business Impact of Leaked Secrets

When a secret is leaked to a public repository or compromised environment, the consequences are immediate and severe. The top three outcomes include:

1. Exposure of Cloud Resources: Attackers gain unauthorized programmatic access to cloud provider consoles (AWS, GCP, Azure), spinning up unauthorized crypto-mining workloads or expensive data storage buckets.
2. Data Theft: Sensitive customer Personally Identifiable Information (PII), proprietary source code, and internal databases are exfiltrated.
3. Account Compromise: Similar to how unauthorized users could once compromise Facebook accounts using simple unverified credentials before Multi-Factor Authentication (MFA) became universal, leaked API keys or GPG keys grant full administrative impersonation.

5. How Trivy Performs Secret Detection

Trivy does not rely on speculative AI or arbitrary guessing to detect secrets. Instead, it utilizes a deterministic, pattern-matching scanning engine.

Detection Mechanisms:

• Regular Expressions (Regex): Scans every line of code against known structural patterns for credentials and tokens.
• Identical Credential Structures: Matches format specifics unique to cloud providers (e.g., AWS access key prefixes like AKIA...).
• High Entropy Strings: Identifies strings with high randomness (such as cryptographic hashes and JWT secrets) that are statistically unlikely to be normal human-readable text.
• Provider-Specific Patterns: Leverages a robust vulnerability database containing regular expressions for GitHub tokens, Google Cloud keys, Slack tokens, and private keys.

6. File Types Scanned by Trivy File System (trivy fs)

Trivy scans a wide variety of file formats in a local directory because secrets can be hardcoded virtually anywhere in a project structure:

• .env Files: Environment variable files containing local configurations.
• YAML / YML Files: Kubernetes manifests, GitHub Actions workflows, and Docker Compose files.
• JSON Files: Application configurations and package registries.
• Python (.py) & Source Files: Application logic containing embedded credentials.
• Dockerfiles: Build instructions containing build-time secrets.
• Shell Scripts (.sh): Deployment scripts containing hardcoded passwords or tokens.

7. Understanding False Positives in Secret Scanning

Because Trivy relies on pattern-matching regex and high-entropy detection, it can occasionally flag items that are not actual security breaches. These are known as false positives.

Common Causes of False Positives:

• Documentation and Examples: Code repositories often contain example files (e.g., .env.example) showing placeholder values like AWS_ACCESS_KEY_ID=your_access_key_here.
• Test Data: Unit tests may contain mock tokens or dummy private keys designed to test authentication logic.
• Placeholder Strings: Generic strings that happen to match the length and entropy profile of a real cryptographic token.

Note: Trivy is conservative by design—it will flag potential secrets and prompt the engineer to manually verify whether a string is a real credential or a harmless example.

What Trivy Does Not Do:

It is important to understand the boundary of Trivy's file system scanner:

• It does not validate secrets: Trivy does not make network calls to cloud providers (e.g., AWS IAM) to test if an API key is currently active.
• It only answers one question: "Does this text string match the known pattern and structural fingerprint of a secret?"

8. Summary and Key Takeaways

• Shift-Left Security: Local file system scanning (trivy fs) allows developers to catch vulnerabilities and secrets before building container images or pushing code to CI/CD pipelines.
• Real-Life Analogy: A secret is like a private diary entry; hardcoding it into code and pushing it to a public repository exposes sensitive data to the entire world.
• Definition of a Secret: Sensitive authentication data including API keys, access tokens, passwords, private SSH/GPG keys, and JSON Web Tokens (JWTs).
• Core Causes of Leaks: Hardcoding to save time, accidentally committing sensitive configuration files, and blindly copying boilerplates.
• Severe Outcomes: Cloud resource exposure, data theft, and full user or system account compromise.
• Pattern-Based Detection: Trivy uses robust regular expressions, provider-specific signatures, and high-entropy string analysis rather than AI guessing.
• Broad File Coverage: Scans .env, YAML, JSON, source code files, Dockerfiles, and shell scripts.
• False Positives: Expected when scanning documentation, examples, and test data; engineers must review flagged items to verify authenticity.
• Non-Intrusive Scanning: Trivy does not connect to cloud providers or validate live keys; it purely evaluates structural text patterns.