Day 10: Python for DevOps — Real-World File Lister & Exception Handling

This chapter focuses on building a practical, production-grade Python utility designed to list files within user-specified directories. By combining lists, loops, the os module, and robust exception handling, we will transition from writing basic scripts to developing resilient, user-friendly DevOps automation tools.

1. The Problem Statement & DevOps Use Case

In DevOps, automation scripts frequently interact with the host operating system's filesystem. A common task is to inspect directories—such as log directories, temporary folders, or build artifact locations—and process the files within them.

The Goal

Develop a Python script that:

1. Accepts a list of folder paths from a user.
2. Iterates through each folder.
3. Lists and prints every file inside those folders.
4. Gracefully handles invalid folder paths or permission issues without crashing.

Input and Output Expectations

2. Designing the Algorithm

To build this program systematically, we break the execution logic down into five distinct steps:

3. Runtime Input via input() vs. Command-Line Arguments

In previous sessions, we explored reading inputs via command-line arguments (sys.argv) and environment variables. Today, we introduce the input() function, which captures user input at runtime (during execution).

Real-Life Analogy: Interactive Prompts

Think of tools like Terraform or APT. When you run terraform apply, the CLI pauses and prompts you with: Do you want to perform these actions? (yes/no): Similarly, running apt install wget prompts you with: Do you want to continue? [Y/n]. The program halts execution, waits for your feedback, and branches its logic based on your response. This is exactly how Python's input() function behaves.

Comparison of Input Mechanisms

Basic input() Demonstration

# test.py
number = input("Please provide a number: ")
print(f"User provided: {number}")

When executed:

$ python test.py
Please provide a number: 5
User provided: 5

4. Converting User Input to Lists via .split()

If we ask a non-technical operator to input a Python list directly (e.g., ['/tmp', '/opt']), they might make syntax errors. To make our script user-friendly, we ask them to provide folder paths separated by spaces, and then programmatically convert that string into a list using the .split() method.

The .split() Method

By default, .split() tokenizes a string using whitespace as the delimiter and returns a list of substrings.

# Example of split conversion
input_string = "/opt /tmp /var/log"
folders_list = input_string.split()

print(folders_list)
# Output: ['/opt', '/tmp', '/var/log']

Implementing Step 1 & 2 in Our Script

# Prompt user for folder paths separated by spaces
user_input = input("Please provide a list of folder names separated by spaces: ")

# Convert the space-separated string into a Python list
folders = user_input.split()

print("Parsed folders list:", folders)

5. Iterating with Loops & Integrating the os Module

To interact with the host operating system, Python provides the built-in os module. Specifically, we use os.listdir(path), which behaves similarly to the shell command ls.

Complexity Note: Listing files in a directory containing $N$ files has a time complexity of $O(N)$ because Python must read each directory entry from the filesystem table.

Naive Implementation (Without Error Handling)

import os

user_input = input("Please provide list of folder names with spaces in between: ")
folders = user_input.split()

for folder in folders:
    print(f"\n--- Listing files for folder: {folder} ---")
    
    # os.listdir returns a list of file names in the directory
    files = os.listdir(folder)
    
    for file in files:
        print(file)

Running the Naive Script with Valid Directories

$ python lister.py
Please provide list of folder names with spaces in between: /opt /tmp

--- Listing files for folder: /opt ---
containerd
python3

--- Listing files for folder: /tmp ---
systemd-private-tmp.log
hsperfdata_root

6. The Naive Implementation & Why It Fails

The naive script works perfectly when all inputs are correct. However, production environments are unpredictable. Let's see what happens if a user provides a non-existent folder path:

$ python lister.py
Please provide list of folder names with spaces in between: /invalid_folder /opt

--- Listing files for folder: /invalid_folder ---
Traceback (most recent call last):
  File "lister.py", line 9, in <module>
    files = os.listdir(folder)
FileNotFoundError: [Errno 2] No such file or directory: '/invalid_folder'

The Problem

The program crashed immediately upon encountering /invalid_folder. It never got the chance to process /opt, even though /opt is a perfectly valid directory. In DevOps, a single bad input should not cause an entire automation pipeline to fail abruptly without processing the remaining valid inputs.

7. Introduction to Exception Handling (try-except)

To prevent crashes, we use Exception Handling. This allows us to catch errors during runtime, log a clean message, and keep the program running.

The try-except Mental Model

• try block: Contains the code that might fail (e.g., reading a directory, dividing numbers, connecting to a database).
• except block: Contains the fallback code that executes only if an error occurs inside the try block.

8. Handling Specific Exceptions (FileNotFoundError & PermissionError)

Instead of catching all errors blindly, it is best practice to catch specific exceptions. This ensures we do not accidentally silence unrelated bugs.

Key OS Exceptions

1. FileNotFoundError: Raised when the specified directory path does not exist.
2. PermissionError: Raised when the script lacks read permissions for the target directory.

Implementing Robust Exception Handling

import os

user_input = input("Please provide list of folder names with spaces in between: ")
folders = user_input.split()

for folder in folders:
    try:
        print(f"\n--- Listing files for folder: {folder} ---")
        files = os.listdir(folder)
        
        for file in files:
            print(file)
            
    except FileNotFoundError:
        print(f"Error: The folder '{folder}' does not exist. Please check the path.")
    except PermissionError:
        print(f"Error: Access denied. You do not have permission to read '{folder}'.")

Execution Test with Mixed Inputs

$ python lister.py
Please provide list of folder names with spaces in between: /invalid_folder /opt

--- Listing files for folder: /invalid_folder ---
Error: The folder '/invalid_folder' does not exist. Please check the path.

--- Listing files for folder: /opt ---
containerd
python3

The script successfully bypassed the invalid folder, printed a clean error message, and proceeded to list the files in /opt.

9. Another Analogy: ZeroDivisionError in a Calculator

To solidify exception handling, let's look at a simple mathematical example. If you write a division utility, dividing any number by zero is mathematically undefined and raises a ZeroDivisionError in Python.

Naive Division vs. Exception-Handled Division

# Naive approach (Crashes if num2 is 0)
num1 = 10
num2 = 0
result = num1 / num2  # Raises ZeroDivisionError

Handled Approach

try:
    num1 = int(input("Enter numerator: "))
    num2 = int(input("Enter denominator: "))
    result = num1 / num2
    print(f"Result: {result}")
except ZeroDivisionError:
    print("Error: Division by zero is not allowed. Please enter a non-zero denominator.")
except ValueError:
    print("Error: Please enter valid integers.")

10. Refactoring to Modular, Production-Ready Code

In professional software development, we avoid writing flat scripts. Instead, we organize our code into reusable functions and use a main entry point. This makes the code modular, testable, and maintainable.

Production-Grade Implementation

import os

def list_files_in_folder(folder_path):
    """
    Lists all files in a given folder path.
    Handles FileNotFoundError and PermissionError gracefully.
    """
    try:
        files = os.listdir(folder_path)
        print(f"\n--- Files in '{folder_path}' ---")
        for file in files:
            print(f"  - {file}")
    except FileNotFoundError:
        print(f"[-] Error: Folder '{folder_path}' does not exist.")
    except PermissionError:
        print(f"[-] Error: Permission denied for folder '{folder_path}'.")

def main():
    """
    Main execution flow.
    """
    user_input = input("Please provide list of folder names with spaces in between: ")
    folders = user_input.split()
    
    for folder in folders:
        list_files_in_folder(folder)

if __name__ == "__main__":
    main()

11. Complete Execution Flow Walkthrough

Here is how data flows through our refactored script during execution:

12. Summary and Key Takeaways

• The input() Function: Captures user input at runtime, pausing program execution until the user presses Enter.
• String Splitting: The .split() method is a clean way to convert a space-separated string of inputs into a structured Python list.
• The os Module: Provides essential operating system interaction utilities, such as os.listdir() to list directory contents.
• Resilient Automation: Exception handling prevents scripts from crashing when encountering minor, expected errors (like a missing directory).
• Targeted Exceptions: Always catch specific errors (e.g., FileNotFoundError, PermissionError) rather than using a generic except: block.
• Modular Design: Structuring scripts with helper functions and a main() entry point makes your DevOps automation clean, readable, and maintainable.