The ReAct Pattern: Reasoning, Acting, and Tool Invocation

This chapter explores the architectural patterns of Agentic AI, focusing on how to design robust, cost-effective, and reliable systems. It covers the Router Pattern for intent classification, the Reflection Pattern for quality control, and a comprehensive decision framework to select the right pattern—such as Chains, Routers, ReAct, Chain of Thought, or Reflection—for production-grade AI applications.

1. The Router Pattern: Dynamic Intent Classification

When building production AI systems, you often encounter diverse user requests where a single, static prompt cannot handle every scenario. The Router Pattern solves this by reading an incoming message, classifying its intent, and dispatching it to a specialized downstream handler.

Unlike sequential chains, a router makes a single, discrete decision to send the request to the most appropriate handler.

Each downstream handler can be designed with an entirely different architecture optimized for its specific task:

• Refund Handler: A deterministic 3-step prompt chain (e.g., verify order $\rightarrow$ check eligibility $\rightarrow$ process refund).
• Complaint Handler: A dynamic ReAct loop that investigates complex, multi-step customer issues.
• FAQ Handler: A single LLM call with retrieved document context (Retrieval-Augmented Generation).

2. Router Implementation Trade-offs: LLM vs. Traditional ML

When implementing a router, developers must choose between using a Large Language Model (LLM) or a traditional Machine Learning (ML) classifier. This choice represents a fundamental trade-off between speed and flexibility.

Real-World Trade-Off Analysis

In high-throughput systems where every millisecond counts, a traditional ML classifier (such as a support vector machine or a lightweight BERT-based model) is preferred. However, if your application must handle highly conversational, ambiguous, or evolving user queries, the semantic understanding of an LLM router justifies the 500 ms latency penalty.

3. Designing Robust Routers: Fallbacks and Category Limits

To deploy a router successfully in production, you must design for failure and manage classification cognitive load.

The Fallback Mechanism

LLMs can occasionally return unexpected classifications or output formats that do not map to your defined handlers. Without a fallback mechanism, your application will crash.

def route_request(user_message: str) -> str:
    # Simulated LLM classification call returning a string category
    intent = call_llm_classifier(user_message) 
    
    # Map of intent categories to their respective handler functions
    handlers = {
        "order": handle_order,
        "refund": handle_refund,
        "complaint": handle_complaint
    }
    
    # Safe fallback: default to the general FAQ handler if intent is unrecognized
    handler = handlers.get(intent, handle_faq)
    return handler(user_message)

Note: Always design your routing logic to fall back to a safe, general-purpose handler (like an FAQ or human-escalation handler) rather than letting the application throw an unhandled exception.

Optimizing Category Counts

The number of categories your router manages directly impacts its accuracy:

• Fewer Categories (4 to 8): High classification accuracy. The LLM easily distinguishes between broad, distinct intents.
• Many Categories (20+): Low classification accuracy. The LLM is highly likely to confuse semantically overlapping categories.

If your business logic requires highly granular routing, implement a two-level routing architecture:

4. The Reflection Pattern: Generator-Evaluator Loop

Some agentic responses carry significantly higher stakes than others. While a minor error in an FAQ response is low-risk, generating incorrect compliance advice or mishandling data privacy obligations can result in severe legal and financial liabilities.

The Reflection Pattern introduces an automated quality check before the response is delivered to the user. It splits the task between two distinct roles:

1. The Generator: Produces the initial response based on the user's prompt.
2. The Evaluator: Evaluates the generated response against strict, testable criteria. If the response fails, the evaluator passes structured feedback back to the generator for a revision.

5. Defining Testable Criteria for Reflection

For the Reflection Pattern to work, the evaluator must use explicit, testable criteria. Vague instructions like "Is this response good?" or "Make sure the tone is polite" are too subjective for an LLM to evaluate consistently.

Examples of Testable vs. Untestable Criteria

If you cannot write your evaluation criteria as a concrete checklist, the Reflection Pattern will fail to yield consistent quality improvements.

6. Code Implementation: Reflect and Correct with Guardrails

When implementing reflection, you must set a strict limit on the maximum number of revision cycles. Without this guardrail, a borderline response can bounce between the generator and evaluator indefinitely, resulting in an infinite loop, high latency, and runaway API costs.

def reflect_and_correct(user_prompt: str, max_revisions: int = 2) -> str:
    # Step 1: Generate initial draft
    response = generate_initial_response(user_prompt)
    
    for attempt in range(max_revisions):
        # Step 2: Evaluate the draft against concrete criteria
        evaluation = evaluate_response(response)
        
        if evaluation.is_valid:
            # Response passed all checks; exit early
            return response
        
        # Step 3: Revise the response using the evaluator's specific feedback
        response = generate_revised_response(
            original_prompt=user_prompt,
            previous_draft=response,
            feedback=evaluation.feedback
        )
        
    # Fallback: Return the final revised response after max attempts are exhausted
    return response

Why Cap Revisions at Two?

In production environments, two revision attempts are typically the sweet spot. If an LLM cannot correct its mistakes within two iterations, a third attempt is statistically unlikely to resolve the issue. At that point, the additional latency and API token costs outweigh any marginal quality improvements.

7. Production Cost Optimization: Selective Reflection

While reflection significantly improves output quality, it is computationally expensive. Each reflection cycle adds $1$ to $2$ additional LLM calls.

Cost Impact: For a customer support agent handling $10,000$ conversations per day, applying reflection to every single message will triple your LLM API costs.

The production solution is Selective Reflection. By combining the Router Pattern with the Reflection Pattern, you can tag incoming interactions by risk level and apply reflection only where it is financially and operationally justified.

By routing low-risk queries directly to simple handlers and reserving the reflection loop for high-stakes compliance or escalated complaints, you maintain rigorous quality assurance where it matters most without scaling your API costs.

8. Common Architectural Mistakes in Agentic Design

When designing agentic systems, engineers frequently fall into several common anti-patterns. Understanding these pitfalls is crucial for passing system design interviews and building viable production systems.

Mistake 1: Using Full Autonomy for a Fixed Process

• The Anti-Pattern: Deploying a fully autonomous ReAct agent to handle a process with static, unchanging steps (e.g., processing an order return).
• The Solution: Use a deterministic Prompt Chain. If a process has four steps that never change, there is no runtime decision-making required. A prompt chain is faster, cheaper, and eliminates the risk of agent deviation.

Mistake 2: Using a Fixed Chain for Dynamic Tasks

• The Anti-Pattern: Forcing an exploratory, unpredictable task (e.g., technical troubleshooting) into a rigid prompt chain.
• The Solution: Use a ReAct Loop with a strict step budget. Technical troubleshooting is highly dependent on user feedback at each step; the agent must be able to dynamically select its next action based on the output of the previous step.

Mistake 3: Overengineering Simple Tasks

• The Anti-Pattern: Building a multi-step pipeline (Intent Classification $\rightarrow$ RAG Retrieval $\rightarrow$ Answer Generation $\rightarrow$ Tone Checking) to answer a simple question like "What are your store hours?"
• The Solution: Use a single LLM call with the FAQ document in context. Overengineering adds $3$ to $4$ seconds of latency and triples costs for zero measurable quality improvement.

Mistake 4: Sending High-Stakes Output Without Review

• The Anti-Pattern: Allowing an agent to send legally binding compliance or data privacy responses directly to a customer without validation.
• The Solution: Add a Reflection Step. While it adds $1$ to $2$ seconds of latency, it prevents catastrophic hallucinations that carry legal and financial liabilities.

Mistake 5: Choosing Patterns Based on Complexity Rather than Fit

• The Anti-Pattern: Assuming that the ReAct pattern is inherently "the best" because it is the most complex and autonomous.
• The Solution: Always start with the simplest pattern that can solve the problem. Complexity expands your application's failure surface, increases debugging difficulty, and introduces latency.

9. The Agentic Pattern Decision Framework

To choose the right pattern for your specific use case, evaluate your task characteristics against the following decision matrix:

10. Summary and Key Takeaways

• The Router Pattern acts as a traffic controller, classifying incoming intents and dispatching them to specialized handlers, optimizing the balance between speed and flexibility.
• LLM Routers vs. ML Classifiers represent a trade-off: LLMs handle highly ambiguous edge cases at the cost of ~500 ms of latency, while traditional ML classifiers run in ~5 ms but require training data and lack flexibility.
• Router Fallbacks are mandatory in production. Always use a default handler (e.g., handlers.get(intent, handle_faq)) to prevent unexpected LLM outputs from crashing the application.
• The Reflection Pattern uses a Generator-Evaluator loop to validate high-stakes outputs against explicit, testable criteria (e.g., checking for PII or word limits).
• Strict Revision Caps (typically max_revisions = 2) prevent infinite loops and control latency and token costs during reflection.
• Selective Reflection optimizes production costs by using a router to tag incoming messages by risk level, applying reflection only to high-stakes or escalated queries.
• The Golden Rule of Agentic Design: Always start with the simplest pattern that works. Introduce complexity (like ReAct or Reflection) only when the dynamic nature or risk level of the task strictly demands it.