Building a Multi-AI Agentic System from Scratch Using LangGraph

A multi-agent architecture coordinates specialized AI agents to handle diverse, complex tasks that exceed the capabilities of a single monolithic prompt. By routing queries dynamically based on intent classification, modern agentic systems can delegate distinct sub-tasks—such as live internet searches, deterministic weather lookups, and programming workflows—to specialized nodes equipped with dedicated domain prompts and tools.

This architectural guide details how to construct a deterministic, multi-agent router workflow using LangGraph, Pydantic structured outputs, ChatGroq (Llama 3.1 70B), and custom LangChain tools.

1. Problem Statement: Moving from Single-Node Q&A to Multi-Agent Architectures

In a basic Question-and-Answer (Q&A) pipeline, all user prompts are sent to a single Large Language Model (LLM) node. While sufficient for generic conversational tasks, this naive pattern fails in production environments requiring specialized capabilities:

Critical Limitations of Single-Node Systems:

• Knowledge Cutoff & Inability to Browse: A standard LLM cannot fetch live information (e.g., current company locations or breaking news) without external retrieval tools.
• Lack of Real-Time Tool Binding: Mathematical or deterministic queries (e.g., real-time weather API lookups) require external functions; feeding all tools to a single LLM inflates prompt tokens and introduces tool-selection hallucination.
• Context Pollution: Domain-specific personas (e.g., expert coding assistants) perform worse when forced to share system prompts with general-purpose web search agents.

The Solution: An Agentic Routing Graph

Instead of relying on a monolithic node, the application first classifies the user's intent, then routes execution to an isolated node or agent explicitly designed for that category before terminating.

2. Core Mental Model: The Router-Agent Workflow Pattern

The system implements a Directed Acyclic Graph (DAG) where the state flows through an evaluation node, reaches a conditional branching router, executes one of three dedicated domain nodes, and exits at a shared termination node.

Flow Lifecycle:

1. Input Ingestion: The graph receives the user query within a shared state container.
2. Intent Classification: A structured output node queries an LLM to categorize the intent into a constrained set of valid strings: coding, Google Search, or weather.
3. Conditional Routing: A conditional edge inspects the classified category and directs execution to the target worker node.
4. Specialized Worker Execution:
   • coding_node: Uses a specialized coding persona prompt.
   • google_node: Executes an agent bound to the Google Serper API wrapper.
   • weather_node: Executes an agent equipped with a custom deterministic tool.
5. State Termination: The chosen node populates the state's answer field and points directly to END.

3. Defining Graph State and Structured Classification Schemas

LangGraph relies on a centralized state passed between every node. We use Pydantic to enforce runtime type safety and strict schema validation.

Schema Implementation

from typing import Literal
from pydantic import BaseModel, Field

# 1. Primary Graph State Schema
class FlowState(BaseModel):
    question: str = Field(description="The original question passed by the user.")
    category: Literal["coding", "Google Search", "weather"] = Field(
        default="Google Search",
        description="The classified category of the user query."
    )
    answer: str = Field(
        default="",
        description="The final response generated by the specialized worker node."
    )

# 2. Structured Output Schema for LLM Intent Extraction
class QuestionCategory(BaseModel):
    category: Literal["coding", "Google Search", "weather"] = Field(
        default="Google Search",
        description="Classified category: 'coding', 'Google Search', or 'weather'."
    )

Note on typing.Literal: Using Literal["coding", "Google Search", "weather"] enforces strict type constraints. If the classification node returns any token outside this set (such as "code" or "Search"), Pydantic raises a validation error before bad state propagates to conditional edges.

4. Building the Intent Classification Node with Structured Outputs

The first operational node in the graph determines which downstream path to take. To avoid brittle regex parsing or JSON string extraction, the LLM is bound directly to our Pydantic schema using .with_structured_output().

from langchain_groq import ChatGroq

# Initialize LLM backend
llm = ChatGroq(model="llama-3.1-70b-versatile")

def check_question_category(state: FlowState) -> FlowState:
    """
    Evaluates the input query and populates state.category with a guaranteed
    member of Literal['coding', 'Google Search', 'weather'].
    """
    structured_llm = llm.with_structured_output(QuestionCategory)
    
    prompt = (
        f"Identify the domain category for the following user question.\n"
        f"Question: {state.question}\n"
        f"If you are unsure, default to 'Google Search'."
    )
    
    response: QuestionCategory = structured_llm.invoke(prompt)
    state.category = response.category
    return state

5. Implementing Dynamic Routing with Conditional Edges

In LangGraph, routing is decoupled from node execution. A routing function reads the state and returns a string identifier that maps to the next node name.

def route(state: FlowState) -> Literal["coding", "Google Search", "weather"]:
    """
    Inspects the updated state category and returns the exact node name to execute next.
    """
    return state.category

Critical Node-Naming Requirement

The strings returned by the routing function (route) must match the identifiers registered via graph.add_node(). If route() returns "Google Search", there must be a node registered as "Google Search".

6. Developing Specialized Worker Nodes and Custom Tools

A. The Coding Worker Node

The coding node does not require external tools; it focuses the LLM on technical problem solving via system prompting.

def coding_node(state: FlowState) -> FlowState:
    """Specialized node for handling programming, algorithm, and software engineering queries."""
    prompt = f"You are an expert programming assistant. Answer the following request clearly:\n{state.question}"
    response = llm.invoke(prompt)
    state.answer = response.content
    return state

B. The Google Search Worker Node

For queries requiring current knowledge (e.g., business locations, recent events), this node runs an agent equipped with Google Serper search capabilities.

from langchain_community.utilities import GoogleSerperAPIWrapper
from langchain.agents import create_react_agent # or custom agent factory

# Configure search tool
search = GoogleSerperAPIWrapper()
search_tools = [search.run]

def create_google_search_agent():
    system_prompt = "You are a search assistant. Look up real-time information on Google to answer user queries accurately."
    return create_react_agent(model=llm, tools=search_tools, state_modifier=system_prompt)

google_agent = create_google_search_agent()

def google_search_node(state: FlowState) -> FlowState:
    """Invokes the search agent and stores the final response in state.answer."""
    agent_input = {
        "messages": [
            {"role": "user", "content": state.question}
        ]
    }
    response = google_agent.invoke(agent_input)
    state.answer = response["messages"][-1].content
    return state

C. The Weather Worker Node with Custom @tool

Custom tools must include a docstring. The underlying agent framework inspects this docstring to decide when and how to format tool calls.

from langchain.tools import tool

@tool
def get_weather(city: str) -> str:
    """
    Fetches the real-time weather and temperature metrics for a specified city.
    """
    # In production, replace with a live API call (e.g., OpenWeatherMap)
    return f"The current temperature in {city} is 23°C with clear skies."

weather_tools = [get_weather]

def create_weather_agent():
    system_prompt = "You are a specialized meteorological assistant capable of providing real-time weather reports."
    return create_react_agent(model=llm, tools=weather_tools, state_modifier=system_prompt)

weather_agent = create_weather_agent()

def weather_node(state: FlowState) -> FlowState:
    """Invokes the weather agent with access to the get_weather tool."""
    agent_input = {
        "messages": [
            {"role": "user", "content": state.question}
        ]
    }
    response = weather_agent.invoke(agent_input)
    state.answer = response["messages"][-1].content
    return state

7. Graph Assembly, Compilation, and Visualization

Once the state, schemas, classification logic, and worker nodes are defined, they are assembled into a compiled StateGraph.

Implementation

from langgraph.graph import StateGraph, START, END

# 1. Initialize Graph
workflow = StateGraph(FlowState)

# 2. Add Nodes
workflow.add_node("check_question_category", check_question_category)
workflow.add_node("coding", coding_node)
workflow.add_node("weather", weather_node)
workflow.add_node("Google Search", google_search_node)

# 3. Add Entrypoint Edge
workflow.add_edge(START, "check_question_category")

# 4. Add Conditional Routing
workflow.add_conditional_edges(
    "check_question_category",
    route,
    {
        "coding": "coding",
        "weather": "weather",
        "Google Search": "Google Search"
    }
)

# 5. Add Terminal Edges
workflow.add_edge("coding", END)
workflow.add_edge("weather", END)
workflow.add_edge("Google Search", END)

# 6. Compile
app = workflow.compile()

Visualizing the Graph

LangGraph supports native graph export using IPython display utilities:

from IPython.display import Image, display

try:
    display(Image(app.get_graph().draw_mermaid_png()))
except Exception:
    # Requires graphviz / mermaid dependencies
    pass

8. End-to-End Execution Walkthrough and Practical Pitfalls

Code Invocation

# Test Case 1: Weather Route
result_weather = app.invoke({"question": "What is the temperature in Indore?"})
print("Result 1 Category:", result_weather["category"])
print("Result 1 Answer:", result_weather["answer"])

# Test Case 2: Google Search Route
result_search = app.invoke({"question": "What is the office address of TechsPlus Learning?"})
print("Result 2 Category:", result_search["category"])
print("Result 2 Answer:", result_search["answer"])

# Test Case 3: Coding Route
result_code = app.invoke({"question": "Write a Python script to reverse a linked list."})
print("Result 3 Category:", result_code["category"])
print("Result 3 Answer:", result_code["answer"])

Common Implementation Pitfalls

1. Node Name Mismatch

• Problem: The router function returns "Google Search", but the node was registered as "google_search".
• Consequence: KeyError: Node 'Google Search' does not exist in graph.
• Fix: Ensure the keys returned by your route function match the first argument of workflow.add_node() exactly.

2. Missing Tool Docstrings

• Problem: A tool function is decorated with @tool but lacks a docstring.
• Consequence: The LLM receives empty tool descriptions, causing argument misattribution or failing to call the tool entirely.
• Fix: Always include clear, explicit docstrings describing the tool's inputs, outputs, and purpose.

3. Loose State Typing

• Problem: Defining category: str instead of Literal[...].
• Consequence: The LLM may return variations like "Code", "coding_task", or "WebSearch", breaking conditional routing.
• Fix: Constrain categories using typing.Literal combined with Pydantic's .with_structured_output().

9. Comparative Architecture: Single LLM vs. Multi-Agent Routing

10. Summary and Key Takeaways

• State-Driven Routing: LangGraph uses a centralized state object (FlowState) that travels across nodes, recording the input query, dynamic category, and final response.
• Structured Intent Classification: Using .with_structured_output() paired with Pydantic models ensures the classification node outputs valid, constrained routing keys.
• Strict Schema Adherence with Literal: Restricting state categories with typing.Literal prevents runtime routing failures caused by naming inconsistencies.
• Separation of Concerns: Domain nodes isolate specialized system prompts and tools (such as SerpAPI for web searches or custom Python tools for weather lookups), reducing context pollution.
• Deterministic Edge Control: add_conditional_edges turns intent classification into programmatic workflow dispatch, ensuring queries only invoke the specific tools they need.
• Production Extensibility: New capabilities (e.g., database queries or document indexing) can be added simply by defining a new node, registering its schema in the Literal type, and mapping it in the conditional edge dictionary.