Building a Conversational Chatbot using LangGraph

An end-to-end guide on architecting stateful conversational AI agents with LangGraph. This chapter covers basic single-node graph topologies, message abstraction typing, conversational reducer functions, multi-turn state loss pitfalls, and implementing conversation persistence via LangGraph checkpointing mechanisms.

1. Architectural Vision & Roadmap

Building production-grade Agentic AI applications requires moving beyond simple sequential or branched scripts into robust, stateful workflows. A full-featured conversational agent integrates multiple advanced subsystems:

• Stateful Conversational Memory: Multi-turn dialogue retention across user interactions.
• Retrieval-Augmented Generation (RAG): Querying domain-specific external knowledge bases and dynamic document sources.
• Tool Calling & Agentic Actions: Allowing the model to invoke external APIs, perform mathematical calculations, or interface with databases.
• Persistence & Checkpointing: Saving execution states to in-memory stores or external databases.
• Human-in-the-Loop (HITL): Enabling human intervention, review, and state editing before critical node execution.
• Fault Tolerance & Retry Logic: Handling API rate limits, node crashes, and network disruptions gracefully.
• Observability: Integrating tracing and evaluation platforms such as LangSmith.

2. Core Mental Model: Chatbot as a Graph Workflow

At its foundational level, an LLM chatbot is a cyclic or iterative execution of a single-node sequential graph.

Breakdown of Execution Flow

1. START Node: Receives the initial payload containing incoming user messages.
2. chat_node (Execution Step):
   • Reads the conversational state.
   • Forwards the accumulated message list to the underlying Large Language Model.
   • Captures the returned response.
   • Appends the new response to the conversational state.
3. END Node: Returns the updated state back to the runtime runner.

The central design consideration in this architecture is the State Schema—the unified data structure tracking the conversation across invocations.

3. Message Abstractions and Schema Representation

When architecting conversational systems in LangGraph, raw Python strings (str) are insufficient for managing structured multi-role conversations. Modern LLMs differentiate context based on distinct message abstractions inheriting from BaseMessage.

Message Types Overview

4. The Role of the Reducer: add_messages vs. Default Overwrites

By default, updating a field in a LangGraph TypedDict state completely overwrites its existing value:

If a user sends an initial message $M_1$, the state is $[M_1]$. When the model generates a response $M_2$, a naive update without a reducer overwrites the state, leaving only $[M_2]$.

To accumulate conversational turns into an append-only log:

LangGraph provides specialized reducers configured via typing.Annotated.

Why add_messages Instead of operator.add?

While Python's built-in operator.add concatenates standard lists, add_messages is built specifically for BaseMessage objects. It handles:

• Appending new messages cleanly.
• Updating existing messages when an incoming message shares an identical unique message ID (in-place replacement for streaming/editing).
• Converting raw message payloads and dictionaries into valid BaseMessage instances.

5. Naive Implementation (Stateless Multi-Turn Pitfall)

The following implementation sets up the single-node chatbot graph using add_messages.

Code Example: Naive Chatbot Graph

from typing import TypedDict, Annotated
from langchain_core.messages import BaseMessage, HumanMessage
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langchain_openai import ChatOpenAI

# 1. Define State Schema
class ChatState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]

# 2. Define Model and Processing Node
llm = ChatOpenAI(model="gpt-4o-mini")

def chat_node(state: ChatState) -> dict:
    messages = state["messages"]
    response = llm.invoke(messages)
    return {"messages": [response]}

# 3. Build and Compile Graph
builder = StateGraph(ChatState)
builder.add_node("chat_node", chat_node)
builder.add_edge(START, "chat_node")
builder.add_edge("chat_node", END)

chatbot = builder.compile()

Implementing a Naive Interactive Terminal Loop

# Interactive Loop
while True:
    user_input = input("Type here: ")
    if user_input.strip().lower() in ["exit", "quit", "by", "bye"]:
        print("Session ended.")
        break

    # Invoking the graph with a single turn
    initial_state = {"messages": [HumanMessage(content=user_input)]}
    result = chatbot.invoke(initial_state)

    # Extracting the last message content
    print(f"AI: {result['messages'][-1].content}")

The "Amnesia" Flaw Explained

When executing this loop across multiple turns:

Turn 1:
User: "Hi, my name is Nitish"
AI: "Hello Nitish! How can I help you today?"

Turn 2:
User: "What is my name?"
AI: "I'm sorry, but I don't have access to your personal information."

Why Does State Loss Occur?

1. Every call to chatbot.invoke(...) creates an isolated execution context.
2. The loop passes only the current message ([HumanMessage(content="What is my name?")]) in initial_state.
3. The previous execution context, along with its populated ChatState, was discarded upon reaching the END node.
4. The LLM receives no prior conversational history and fails context-dependent queries.

6. Solving State Loss: LangGraph Persistence & Checkpointers

To maintain conversational history across distinct .invoke() boundaries without manually managing message arrays in the application layer, LangGraph uses Checkpointers.

Checkpointer Architecture

A Checkpointer automatically writes the state to storage at each step of the graph execution.

Storage Backends: In-Memory vs. Production Databases

7. Multi-Turn Stateful Architecture Implementation

Complete Working Implementation with MemorySaver

from typing import TypedDict, Annotated
from langchain_core.messages import BaseMessage, HumanMessage
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.checkpoint.memory import MemorySaver
from langchain_openai import ChatOpenAI

# 1. State Definition
class ChatState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]

# 2. Node Logic
llm = ChatOpenAI(model="gpt-4o-mini")

def chat_node(state: ChatState) -> dict:
    response = llm.invoke(state["messages"])
    return {"messages": [response]}

# 3. Graph Assembly with Checkpointing
builder = StateGraph(ChatState)
builder.add_node("chat_node", chat_node)
builder.add_edge(START, "chat_node")
builder.add_edge("chat_node", END)

# Instantiate memory checkpointer and compile
checkpointer = MemorySaver()
chatbot = builder.compile(checkpointer=checkpointer)

# 4. Multi-Turn Interactive Execution Loop
thread_config = {"configurable": {"thread_id": "session_alpha"}}

while True:
    user_input = input("Type here: ").strip()
    if user_input.lower() in ["exit", "quit", "by", "bye"]:
        print("Session ended.")
        break
    
    if not user_input:
        continue

    # Invoke graph using the persistent thread configuration
    state_payload = {"messages": [HumanMessage(content=user_input)]}
    result = chatbot.invoke(state_payload, config=thread_config)

    latest_response = result["messages"][-1].content
    print(f"AI: {latest_response}")

Verifying Internal Checkpoint State

The state can be queried directly using get_state() with the corresponding thread_id:

current_state = chatbot.get_state(thread_config)
print("Persisted Message Count:", len(current_state.values["messages"]))
for msg in current_state.values["messages"]:
    print(f"[{msg.type.upper()}]: {msg.content}")

8. Multi-Thread Isolation & Session Partitioning

Conversations must be segmented per user or session. LangGraph enforces this isolation via thread_id inside the config payload.

If User B queries the agent using thread_id: "user_session_2", they cannot access the conversational state or memory of user_session_1.

Summary and Key Takeaways

• Graph Topology Simplicity: A basic conversational chatbot runs as a single-node workflow (START -> chat_node -> END), where the complexity lies in managing state transitions across multiple runs.
• Typed Message Objects: Using BaseMessage models (HumanMessage, AIMessage, SystemMessage, ToolMessage) preserves semantic context better than plain strings.
• State Reducers: The add_messages reducer prevents state overwrites by appending new turns and managing message updates via unique IDs.
• Root Cause of Conversational Amnesia: Calling .invoke() repeatedly in a loop without a persistence layer initializes a fresh execution context each time, discarding prior conversation history.
• Checkpointers: Checkpointers like MemorySaver capture the current state upon reaching the END node and automatically load previous dialogue turns into the graph's working state on the next invocation.
• Thread-Based Concurrency: Supplying a thread_id under {"configurable": {"thread_id": "..."}} partitions conversations, ensuring multiple users can interact with the same compiled graph independently.