AI Engineer In 10 Weeks | Episode 09: Prompt Chaining

Prompt Chaining is a fundamental architectural pattern in production AI engineering. Instead of relying on a single, massive prompt to handle complex business logic, Prompt Chaining breaks execution into modular, atomic sub-tasks. By feeding the output of one LLM call directly as the input into the next, developers gain explicit control over workflow state, improve system debuggability, lower operating costs, and dramatically increase output accuracy.

1. Student Doubts & Foundations: ReAct vs. Prompt Chaining

Before diving into Prompt Chaining, it is vital to clarify common misconceptions regarding agentic patterns and syntax practice.

ReAct (Reasoning + Action) vs. Implementation

• ReAct is a Conceptual Framework: ReAct defines how an LLM reasons through a task and selects tool execution dynamically. It is a mental model for AI agents, not something you manually code from scratch in business applications.
• LLM-Driven Execution: Modern LLMs natively handle tool selection when supplied with prompt instructions and JSON schema tool definitions.
• Interview Context: Technical interviews test your conceptual understanding of ReAct (how LLMs decide what, when, and how to call available tools). You will rarely be asked to code a raw ReAct loop from scratch.

Effective Code Learning Mindset

• Avoid Copy-Pasting: Viewing code or reading notes passively creates a false sense of fluency. Write code line-by-line.
• Syntax Mastery Takes Time: Programming syntax requires deliberate repetition. Use LLMs like ChatGPT to explain unfamiliar syntax, but type out solutions manually to build muscle memory.
• Iterative Modification: Take lecture code and deliberately modify prompts, introduce edge cases, or extend functional logic.

2. The Problem with Monolithic Prompts in Production

In non-production environments, single-prompt interactions appear sufficient. You pass a prompt to an LLM, and it returns a decent response. However, enterprise workflows (e.g., resume screeners, legal analysis, automated reporting) involve complex, multi-stage logic.

The Naive Monolithic Prompt Approach

A developer writes a single massive prompt detailing every single business requirement and feeds it to the LLM in one giant request.

Key Failures of Monolithic Prompts

1. Instruction Degradation: LLMs struggle to pay equal attention to dozens of instructions in a single context window (the "lost in the middle" phenomenon).
2. Brittle Debugging: If the final output is wrong, it is impossible to determine which internal reasoning step failed.
3. Inflexible Model Selection: You are forced to run simple sub-tasks (like text formatting) on expensive, high-end models.
4. All-or-Nothing Failures: If the final step fails, the entire workflow must be re-executed from scratch, wasting API tokens and execution time.

3. Real-Life Analogy: The House Building Workflow

Real-Life Analogy

Consider the workflow required to construct and deliver a complete house:

• The Monolithic Approach: You hire a single contractor, hand them a massive list of tasks, and tell them: "Buy the land, draft the architectural blueprints, clear government permits, dig the foundation, lay bricks, install plumbing, and paint the walls." If the resulting building is structurally unsafe, identifying whether the flaw originated from bad land selection, bad engineering, or poor bricklaying is nearly impossible.
• The Prompt Chaining Approach: You break construction down into sequential, specialized sub-tasks:
  1. Phase 1 (Land Selection): Buy a suitable plot -> Outputs plot coordinates.
  2. Phase 2 (Architectural Design): Pass plot coordinates to an architect -> Outputs blueprints.
  3. Phase 3 (Structural Construction): Pass blueprints to a structural engineer -> Outputs physical frame.
  4. Phase 4 (Finishing): Pass physical frame to a painter -> Outputs completed house.

4. Architectural Concept: What is Prompt Chaining?

Prompt Chaining is a software architecture pattern where a complex task is split into a sequence of small, highly targeted prompts. Each prompt executes an atomic sub-task using an LLM call, and its output is programmatically captured, transformed if necessary, and passed as input into the subsequent prompt.

5. The Four Pillars of Prompt Chaining

Deep-Dive: Debugging and Multi-Model Optimization

Scenario: The Debugging Workflow

Imagine a candidate named Pratyush Narayan submits a high-quality resume matching a Job Description (JD), but the automated system returns a rejection score of 30/100.

• Under a Monolithic System: You must read through pages of generated raw text to guess why the model gave a low score.
• Under a Prompt Chain System: You inspect the modular outputs step-by-step:
  • Step 1 Output (Candidate Skills): ["Python", "FastAPI", "Docker", "Git"] -> Pass
  • Step 2 Output (JD Skills): ["Python", "FastAPI", "Docker", "Git"] -> Pass
  • Step 3 Output (Matching Score): 30 -> Fail (Bug pinpointed in Step 3 scoring prompt logic).

Scenario: Multi-Model Cost Optimization

Not all sub-tasks require state-of-the-art LLMs:

• Complex Task: Parsing structured entities out of unstructured, multi-page PDFs -> Use Flagship LLM (e.g., Claude 3.5 Sonnet / GPT-4o).
• Simple Task: Comparing two lists of JSON strings or drafting a standardized email -> Use Lightweight/Free LLM (e.g., GPT-3.5-Turbo / Llama-3-8B / Groq endpoints).

6. Prompt Chaining vs. ReAct Pattern

It is crucial to understand the architectural distinction between Prompt Chaining and ReAct agents:

7. Workflow Walkthrough: Automated Resume Parser

To illustrate Prompt Chaining, we analyze a multi-step Resume Parsing pipeline that evaluates a candidate against a Job Description.

1. Step 1 (Resume Skill Extraction): Extracts raw skills from candidate's resume string.
2. Step 2 (JD Skill Extraction): Extracts required and preferred skills from the job description string.
3. Step 3 (Skill Matching & Scoring): Compares extracted Candidate Skills against JD Skills, returning a score (1–100) and rationale.
4. Step 4 (Programmatic Decision Routing): If score >= 60, route to HR scheduling; otherwise, generate a standard rejection notification.

8. Python Implementation: Step-by-Step Walkthrough

Below is the complete, runnable Python implementation demonstrating Prompt Chaining with modular helper functions, strict system prompts, and rate-limiting controls.

import time
import os
from openai import OpenAI

# Initialize the LLM client (using OpenAI API or compatible Groq/Local endpoint)
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY", "your-api-key"))

# ==========================================
# GLOBAL INPUT DATA
# ==========================================
JOB_DESCRIPTION = """
We are seeking a Backend AI Engineer with 2+ years of experience.
Required Skills: Python, FastApi, Django, PostgreSQL, Docker, AWS, REST APIs, Git.
Nice to have: Vector Databases, LangChain.
"""

CANDIDATE_RESUME = """
Name: Rahul Sharma
Experience: 3 Years in Software Engineering.
Skills & Tools: Proficient in Python, FastApi, MySQL, Docker, REST APIs, Git.
Built multiple backend microservices and handled deployment pipelines.
"""

# ==========================================
# GENERIC LLM CALL HELPER
# ==========================================
def ask_llm(system_prompt: str, user_prompt: str, model: str = "gpt-4o-mini") -> str:
    """
    Generic wrapper for making structured completion calls to the LLM.
    """
    system_message = {"role": "system", "content": system_prompt}
    user_message = {"role": "user", "content": user_prompt}
    
    response = client.chat.completions.create(
        model=model,
        messages=[system_message, user_message],
        temperature=0.0
    )
    return response.choices[0].message.content.strip()

# ==========================================
# CHAIN STEP 1: RESUME SKILL EXTRACTION
# ==========================================
def step1_extract_resume_skills(resume_text: str) -> str:
    system_prompt = (
        "You are a professional HR assistant. Extract all technical skills from the "
        "provided resume. Return ONLY a clean, comma-separated list of skills. "
        "Do NOT include conversational text or invent skills not explicitly mentioned."
    )
    # CRITICAL: Use f-string to inject the variable
    user_prompt = f"Extract skills from this candidate resume:\n{resume_text}"
    return ask_llm(system_prompt, user_prompt)

# ==========================================
# CHAIN STEP 2: JD SKILL EXTRACTION
# ==========================================
def step2_extract_jd_skills(jd_text: str) -> str:
    system_prompt = (
        "You are a professional HR assistant. Extract all required technical skills "
        "from the job description. Return ONLY a clean, comma-separated list of skills. "
        "Do NOT include extra commentary or conversational filler."
    )
    user_prompt = f"Extract required skills from this job description:\n{jd_text}"
    return ask_llm(system_prompt, user_prompt)

# ==========================================
# CHAIN STEP 3: SKILL MATCHING & SCORING
# ==========================================
def step3_match_and_score(candidate_skills: str, jd_skills: str) -> str:
    system_prompt = (
        "You are an expert HR evaluation engine. Compare the candidate's skills with "
        "the job description requirements. Provide:\n"
        "1. Skill Match Breakdown (Matched vs Missed skills)\n"
        "2. A numeric fit score between 1 and 100\n"
        "3. A concise fit verdict."
    )
    user_prompt = (
        f"Candidate Skills: {candidate_skills}\n"
        f"Job Description Required Skills: {jd_skills}\n\n"
        "Perform comparison and generate score now."
    )
    return ask_llm(system_prompt, user_prompt)

# ==========================================
# MAIN CHAIN EXECUTION PIPELINE
# ==========================================
def run_prompt_chain():
    print("--- STARTING PROMPT CHAIN WORKFLOW ---\n")
    
    # Step 1: Extract candidate skills
    print("[Executing Step 1: Extracting Candidate Skills...]")
    candidate_skills = step1_extract_resume_skills(CANDIDATE_RESUME)
    print(f"-> Candidate Skills: {candidate_skills}\n")
    time.sleep(2)  # Prevent rate limits between consecutive API calls
    
    # Step 2: Extract JD skills
    print("[Executing Step 2: Extracting JD Requirements...]")
    jd_skills = step2_extract_jd_skills(JOB_DESCRIPTION)
    print(f"-> JD Required Skills: {jd_skills}\n")
    time.sleep(2)
    
    # Step 3: Match and Score
    print("[Executing Step 3: Skill Matching & Final Evaluation...]")
    evaluation = step3_match_and_score(candidate_skills, jd_skills)
    print(f"-> Evaluation Result:\n{evaluation}\n")
    
    print("--- PROMPT CHAIN WORKFLOW COMPLETED ---")

if __name__ == "__main__":
    run_prompt_chain()

9. Real-World Debugging & Production Pitfalls

During live implementation, several failure modes frequently occur in Prompt Chains. Understanding how to debug them is vital for production systems.

Pitfall 1: Forgotten Python F-Strings

• The Bug: Writing user_prompt = "Extract skills from: {resume_text}" without the f prefix.
• Symptom: The string literal "{resume_text}" is sent to the API instead of the actual resume content. The LLM responds: "No resume content was provided."
• Fix: Ensure all dynamic variables are wrapped in formatted string literals (f"...").

Pitfall 2: Conversational Fluff In Intermediate Outputs

• The Bug: Step 1 returns: "Sure! Here are the extracted skills: Python, FastApi, Docker."
• Symptom: Downstream step parsers break because of extra preamble text.
• Fix: Enforce strict output constraints in the System Prompt:

  "Return ONLY a clean, comma-separated list of skills. Do NOT return conversational intro text or extra filler."

Pitfall 3: API Rate Limits (HTTP 429)

• The Bug: Executing multiple LLM calls back-to-back inside loops without throttling.
• Symptom: API provider rejects the 2nd or 3rd call due to Requests Per Minute (RPM) thresholds.
• Fix: Inject explicit pauses (time.sleep(2)) between pipeline stages or implement exponential backoff retry wrappers.

10. Summary and Key Takeaways

• Prompt Chaining defined: Decomposing complex tasks into an explicit, sequential pipeline of atomic LLM calls where output $N$ feeds input $N+1$.
• Deterministic vs. Autonomous: Unlike ReAct agents where the LLM decides actions dynamically, Prompt Chaining gives developers full programmatic control over execution order.
• Granular Debuggability: When system outputs fail, chaining allows developers to inspect intermediate step results and fix specific prompts without rewriting the whole pipeline.
• Multi-Model Cost Efficiency: Allows pairing expensive LLMs with difficult sub-tasks (e.g., layout parsing) and cheap/fast LLMs with low-complexity sub-tasks (e.g., list formatting).
• System Prompt Enforcement: Always restrict intermediate step outputs (e.g., requesting strict JSON or comma-separated lists) to prevent downstream parser failures.
• API Throttling Safety: Intersperse consecutive API steps with rate-limiting pauses (time.sleep) or retry handlers to maintain pipeline stability.
• Practical Learning Strategy: Master code execution by building line-by-line, isolating failure states step-by-step, and tuning prompt behavior incrementally.