End-to-End RAG Chatbot with PDF Upload | Streamlit + LangChain + Agentic RAG

An end-to-end Retrieval-Augmented Generation (RAG) system bridges static Large Language Models (LLMs) with dynamic, domain-specific private data. By framing retrieval not as a rigid, hardcoded chain but as an autonomous tool within an Agentic RAG architecture, the model dynamically decides when to consult external documents and when to answer directly from parametric memory.

This guide details the complete process of building an enterprise-grade QA chatbot using LangChain, LangGraph, and Streamlit. It covers everything from multi-file PDF ingestion to vector search debugging, stateful conversational memory, and reactive user interface (UI) rendering.

1. Architectural Overview: From Standard RAG to Agentic RAG

In traditional linear RAG architectures, every user prompt—regardless of its context or intent—triggers a vector database search. If a user simply enters "Hello" or "Thank you", the system wastes compute running cosine similarity algorithms over document chunks.

Structure & Concept Flow

Linear RAG Workflow:

User Query -> Embed Query -> Force Vector DB Search -> Stuff Chunks -> LLM -> Response

Agentic RAG shifts the paradigm by making the retrieval mechanism an on-demand tool available to an autonomous reasoning engine:

Structure & Concept Flow

Agentic RAG Workflow:

User Query

LLM Agent (Reasoning Engine)

+---> Decides context is NOT needed (e.g., "Hello") ----> Direct Answer

+---> Decides context IS needed ------------------------> Calls Retrieve Tool

Vector Database

Returns Top-K Chunks

LLM Synthesizes Answer

Key Differences

Architectural Layer	Standard / Linear RAG	Agentic RAG
Control Flow	Fixed, sequential pipeline (`Chain`)	Dynamic, self-directed loop (`Agent`)
Tool Execution	Deterministic (always queries DB)	Autonomous (invokes tool conditionally via Docstrings)
Conversational Flow	Chained prompt stuffing	Iterative ReAct loop with episodic/session memory
Failover / Edge Handling	Inflexible; fails on out-of-domain conversational queries	Gracefully routes greetings, clarifications, and external lookups

2. Environment Setup and Library Dependencies

To support file ingestion, embedding models, vector storage, agent graph runtimes, and a web interface, several decoupled packages are required.

BASH

pip install -q \
    python-dotenv \
    pypdf \
    langchain \
    langchain-community \
    langchain-openai \
    langchain-groq \
    langgraph \
    streamlit

Essential Library Roles

pypdf: Low-level binary PDF stream parsing.
langchain_community.document_loaders: High-level wrappers (PyPDFLoader, PyPDFDirectoryLoader) that extract raw text and page-level metadata.
langchain_text_splitters: Splits text into semantic segments to stay within embedding model context limits.
langchain_openai: High-dimensional embedding generation (text-embedding-3-large).
langchain_groq: High-throughput LLM runtime acceleration (e.g., Llama-3, Mixtral, or Groq-hosted open-weights models).
langgraph: Orchestration runtime handling cyclic agent execution paths and conversational checkpoint persistence.
streamlit: Front-end reactive web application framework.

3. Naive CLI Implementation: Ingestion and Retrieval Pipeline

Before wrapping the solution in an interactive UI, the core indexing and retrieval pipeline must be validated via a deterministic command-line interface (CLI).

Structure & Concept Flow

Document Ingestion Pipeline:

Medical Report (PDF)

| PyPDFLoader loads raw pages

Raw Document Objects [doc.page_content, doc.metadata]

| RecursiveCharacterTextSplitter (chunk_size=1000, overlap=200)

Text Chunks ($N$ segmented nodes)

| OpenAIEmbeddings (text-embedding-3-large)

Vector Embeddings ($\mathbb{R}^{3072}$)

In-Memory Vector Store (InMemoryVectorStore)

Step 1: Loading and Splitting Documents

PYTHON

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import InMemoryVectorStore

load_dotenv()

# Load specific document
loader = PyPDFLoader(file_path="data/medical_report.pdf")
raw_docs = loader.load()

# Split text into manageable overlapping segments
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200
)
chunked_docs = text_splitter.split_documents(documents=raw_docs)

Chunking Parameters:

chunk_size=1000: Caps each text fragment at 1,000 characters, fitting comfortably within the model's context window.

chunk_overlap=200: Retains 200 characters of context between adjacent chunks, preventing sentences or ideas from being cut off mid-thought.

Step 2: Vector Embedding and Indexing

Text chunks are transformed into dense vector representations ( $\mathbb{R}^d$ ) and indexed in an ephemeral memory store:

PYTHON

# Initialize high-dimensional embedding engine
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")

# Index documents in memory using cosine similarity
vector_db = InMemoryVectorStore.from_documents(
    documents=chunked_docs,
    embedding=embeddings
)

4. Building the Retrieval Tool and Agent Reasoning Loop

To give an agent access to external data, we expose the vector store as a functional tool using LangChain's @tool decorator. The tool's docstring serves as an instruction manual for the LLM's reasoning engine.

PYTHON

from langchain_core.tools import tool
from langchain_groq import ChatGroq

@tool
def retrieve_context(query: str) -> str:
    """
    Retrieve documents relevant to a user query from the uploaded knowledge base.
    Always invoke this tool when answering domain-specific, factual, or medical report inquiries.
    """
    # Retrieve top K most similar chunks
    matched_docs = vector_db.similarity_search(query=query, k=3)
    
    context = ""
    for doc in matched_docs:
        context += doc.page_content + "\n\n"
    
    return context.strip()

Similarity Search Mechanics

The vector database calculates the cosine similarity between the query vector $\mathbf{u}$ and candidate chunk vectors $\mathbf{v}$ :

\text{Cosine Similarity} = \cos(\theta) = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\|_2 \|\mathbf{v}\|_2} = \frac{\sum_{i=1}^{d} u_i v_i}{\sqrt{\sum_{i=1}^{d} u_i^2} \sqrt{\sum_{i=1}^{d} v_i^2}}

The retrieval engine ranks chunks by their similarity score and returns the top $k=3$ results, which are then passed into the LLM's generation prompt.

PYTHON

# System prompt to anchor agent behavior
system_prompt = (
    "You are a helpful assistant that answers questions using retrieved context. "
    "Your knowledge base consists of details from the uploaded document. "
    "Always use the retrieve_context tool for questions requiring external knowledge. "
    "If the answer cannot be found in the context, state that clearly."
)

# Initialize fast-inference LLM
llm = ChatGroq(model="openai/gpt-oss-20b")

5. Integrating Conversation Memory via LangGraph Checkpointers

Stateless LLM interactions cannot maintain multi-turn context (such as resolving anaphoric references like "What is her age?" after asking "What is the patient's name?"). We maintain conversational state by integrating an InMemorySaver checkpointer using LangGraph.

Structure & Concept Flow

Stateful Graph Workflow:

User Input ("What is her age?")

| Config: {"configurable": {"thread_id": "session_001"}}

Checkpointer (Loads state for "session_001")

Agent Evaluates History:

["User: What is the patient's name?", "AI: The patient is Nikita.", "User: What is her age?"]

Infers "her" = "Nikita" -> Invokes retrieve_context("Nikita age")

Checkpointer saves state with new AI response

Implementing Agent Memory

PYTHON

from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import InMemorySaver

# Initialize ephemeral in-memory state saver
memory = InMemorySaver()

# Compile the ReAct agent with tools, model, and memory checkpointer
agent = create_react_agent(
    model=llm,
    tools=[retrieve_context],
    state_modifier=system_prompt,
    checkpointer=memory
)

6. Runtime Debugging: Real-World Failure Modes and Resolutions

Building an agentic RAG system involves debugging integration points across several decoupled libraries. The following issues are commonly encountered when setting up the pipeline:

Structure & Concept Flow

Runtime Debugging Checkpoints:

Vector DB returns empty strings?

+---> Check embedding model dimensions (e.g., text-embedding-3-small vs large mismatch)

Agent execution throws Config Error?

+---> Ensure invoke() includes {"configurable": {"thread_id": "<id>"}}

CompiledGraph is not callable?

+---> Replace agent() direct call with agent.invoke(...)

1. The Checkpointer Configuration Error

Symptom: TypeError: checkpointer requires one or more configurable keys.
Root Cause: Invoking a stateful LangGraph agent without specifying a conversation thread identifier.
Resolution: Always pass a configurable dictionary mapping a unique thread_id:

PYTHON

# Incorrect invocation
response = agent.invoke({"messages": [{"role": "user", "content": "Hello"}]})

# Correct invocation
response = agent.invoke(
    {"messages": [{"role": "user", "content": "Hello"}]},
    config={"configurable": {"thread_id": "default_user"}}
)

2. Semantic Misalignment & Embedding Mismatches

Symptom: The retrieval tool is called, but returns irrelevant context or fails to find obvious facts (e.g., missing the patient's name).
Root Cause: Using low-dimensional or mismatched embedding variants (text-embedding-3-small) where fine-grained medical terms fall below the cosine similarity threshold.
Resolution: Switch to text-embedding-3-large and run targeted debugging to verify chunk contents:

PYTHON

# Diagnostic script to isolate ingestion failures
debug_docs = vector_db.similarity_search("patient name", k=1)
print(f"DEBUG: Retrieved Chunks Count = {len(debug_docs)}")
print(f"DEBUG: Top Chunk Content = {debug_docs[0].page_content}")

3. Callable Graph Runtime Exceptions

Symptom: TypeError: 'CompiledStateGraph' object is not callable.
Root Cause: Treating the compiled agent instance as a standard Python function (e.g., agent(...)).
Resolution: Call the .invoke() method directly on the compiled graph: agent.invoke(...).

7. Transitioning to a Production-Ready Streamlit Interface

Moving from a local terminal loop to a multi-user web application requires shifting from synchronous, blocking scripts to a reactive execution model.

Structure & Concept Flow

Streamlit Execution Cycle:

User Interaction (Click, File Upload, Text Entry)

Streamlit re-executes entire python file from line 1

Local variables are cleared unless stored in st.session_state

To deliver a clean user experience, the interface is split into two mutually exclusive states:

Document Ingestion View: Displays an upload button, drag-and-drop zone, and processing spinner. Shown while no document is active.
Interactive Chat View: Displays conversational message bubbles and a fixed input bar at the bottom. Rendered once the vector store and agent are initialized.

CODE

State Machine Logic:
if not st.session_state.document_uploaded:
    Render File Uploader
else:
    Render Chat History
    Render Chat Input Form

8. State Management: Handling Script Re-runs with `st.session_state`

Because Streamlit scripts run from top to bottom on every user interaction, storing objects like the vector database or agent instance in local variables would trigger an expensive re-index on every message.

We avoid this by initializing these objects inside st.session_state, ensuring they persist across re-renders:

PYTHON

import streamlit as st

# Initialize session state flags and data structures
if "document_uploaded" not in st.session_state:
    st.session_state.document_uploaded = False

if "agent" not in st.session_state:
    st.session_state.agent = None

if "vector_store" not in st.session_state:
    st.session_state.vector_store = None

if "messages" not in st.session_state:
    st.session_state.messages = []

Session State Variable Schema

Key	Type	Lifecycle	Purpose
`document_uploaded`	`bool`	Persists across interactions	Toggles visibility between upload and chat interfaces
`agent`	`CompiledStateGraph`	Instantiated once on upload	Holds the compiled LangGraph ReAct agent
`vector_store`	`InMemoryVectorStore`	Instantiated once on upload	Retains vector embeddings in application memory
`messages`	`list[dict]`	Grows on every message	Maintains UI message history: `[{"role": "...", "content": "..."}]`

9. Dynamic Multi-PDF Document Ingestion Pipeline

To support realistic workloads, the application must handle multiple PDF uploads at once rather than relying on a single static file path.

PyPDFLoader is designed for single files. To process an entire directory of newly uploaded documents in one pass, we use PyPDFDirectoryLoader.

Structure & Concept Flow

Multi-File Ingestion Pipeline:

User selects files: ['report1.pdf', 'report2.pdf']

Iterate through uploaded files & write byte streams to apps/doc_files/

PyPDFDirectoryLoader scans and parses apps/doc_files/

Split into text chunks via RecursiveCharacterTextSplitter

Embed via OpenAIEmbeddings -> Store in InMemoryVectorStore

Construct Agent Tool -> Instantiate ReAct Agent

Save agent to st.session_state.agent -> st.rerun()

Ingestion and Agent Construction Code

PYTHON

import os
from langchain_community.document_loaders import PyPDFDirectoryLoader

def process_documents(directory_path: str):
    """
    Parses all PDFs in the target directory, splits them into chunks,
    generates embeddings, and initializes the ReAct agent.
    """
    # 1. Ingest all PDFs in the directory
    dir_loader = PyPDFDirectoryLoader(path=directory_path)
    raw_documents = dir_loader.load()

    # 2. Split documents into semantic chunks
    splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
    split_docs = splitter.split_documents(raw_documents)

    # 3. Create embeddings and index vectors
    embedding_model = OpenAIEmbeddings(model="text-embedding-3-large")
    vector_db = InMemoryVectorStore.from_documents(
        documents=split_docs,
        embedding=embedding_model
    )
    st.session_state.vector_store = vector_db

    # 4. Define retrieval tool bound to this vector store
    @tool
    def retrieve_context(query: str) -> str:
        """Retrieve context from documents in the uploaded directory."""
        results = vector_db.similarity_search(query=query, k=3)
        return "\n\n".join([doc.page_content for doc in results])

    # 5. Initialize LLM and stateful agent
    llm = ChatGroq(model="openai/gpt-oss-20b")
    memory = InMemorySaver()
    
    agent_executor = create_react_agent(
        model=llm,
        tools=[retrieve_context],
        state_modifier=(
            "You are an assistant answering questions using the retrieve_context tool. "
            "Base your answers exclusively on the retrieved context."
        ),
        checkpointer=memory
    )
    
    # Update persistent application state
    st.session_state.agent = agent_executor
    st.session_state.document_uploaded = True

10. Message History and Chat UI Rendering

Streamlit's UI rendering pipeline follows a simple structure:

Re-render historical messages from st.session_state.messages.
Accept new input using st.chat_input.
Append user input to history, display it, run the agent, display the response, and store the result.

PYTHON

# Render historical messages on re-render
for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.markdown(msg["content"])

# Collect and process new user input
if prompt := st.chat_input("Ask a question about your uploaded documents:"):
    # 1. Append user message to history and render it
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    # 2. Invoke the stateful agent
    with st.chat_message("assistant"):
        agent_input = {"messages": [{"role": "user", "content": prompt}]}
        agent_config = {"configurable": {"thread_id": "streamlit_session"}}
        
        raw_response = st.session_state.agent.invoke(agent_input, config=agent_config)
        ai_reply = raw_response["messages"][-1].content
        
        st.markdown(ai_reply)

    # 3. Append assistant response to history
    st.session_state.messages.append({"role": "assistant", "content": ai_reply})

11. End-to-End System Walkthrough

Below is the complete, runnable Python script (app.py), integrating the dynamic file ingestion pipeline, session management, and conversational UI.

PYTHON

import os
import streamlit as st
from dotenv import load_dotenv

# LangChain, LangGraph, and Provider Imports
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import InMemoryVectorStore
from langchain_core.tools import tool
from langchain_groq import ChatGroq
from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import InMemorySaver

# Load API credentials from .env
load_dotenv()

# App-level UI Configuration
st.set_page_config(page_title="Enterprise Agentic RAG", layout="centered")

# --- 1. Session State Initialization ---
if "document_uploaded" not in st.session_state:
    st.session_state.document_uploaded = False

if "agent" not in st.session_state:
    st.session_state.agent = None

if "vector_store" not in st.session_state:
    st.session_state.vector_store = None

if "messages" not in st.session_state:
    st.session_state.messages = []

# --- 2. Ingestion and Processing Pipeline ---
def process_documents(directory_path: str):
    """Loads directory PDFs, builds vector index, and compiles the agent."""
    dir_loader = PyPDFDirectoryLoader(path=directory_path)
    raw_docs = dir_loader.load()

    splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
    chunked_docs = splitter.split_documents(raw_docs)

    embedding_engine = OpenAIEmbeddings(model="text-embedding-3-large")
    vector_store = InMemoryVectorStore.from_documents(
        documents=chunked_docs, 
        embedding=embedding_engine
    )
    st.session_state.vector_store = vector_store

    @tool
    def retrieve_context(query: str) -> str:
        """Search the indexed vector store for context matching the query."""
        matched = vector_store.similarity_search(query=query, k=3)
        return "\n\n".join([doc.page_content for doc in matched])

    llm = ChatGroq(model="openai/gpt-oss-20b")
    memory = InMemorySaver()

    system_prompt = (
        "You are a helpful assistant answering user questions using retrieved context. "
        "Always call retrieve_context for inquiries about specific facts or reports. "
        "Keep responses factual, professional, and clear."
    )

    agent_executor = create_react_agent(
        model=llm,
        tools=[retrieve_context],
        state_modifier=system_prompt,
        checkpointer=memory
    )

    st.session_state.agent = agent_executor
    st.session_state.document_uploaded = True

# --- 3. View A: File Upload UI ---
if not st.session_state.document_uploaded:
    st.title("Document Upload & Ingestion")
    st.markdown("Upload domain documents (e.g., medical records, technical manuals) to initialize the assistant.")

    uploaded_files = st.file_uploader(
        label="Select PDF files",
        type=["pdf"],
        accept_multiple_files=True
    )

    if uploaded_files:
        with st.spinner("Processing files and building index..."):
            storage_dir = os.path.join("apps", "doc_files")
            os.makedirs(storage_dir, exist_ok=True)

            # Save uploaded byte streams to the storage directory
            for file in uploaded_files:
                target_path = os.path.join(storage_dir, file.name)
                with open(target_path, "wb") as f:
                    f.write(file.get_value())

            # Run indexing pipeline
            process_documents(storage_dir)

        # Trigger reactive re-run to transition to Chat UI
        st.rerun()

# --- 4. View B: Conversational Chat UI ---
else:
    st.title("Domain-Specific Assistant")

    # Render previous conversation history
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # Accept new conversational input
    if user_query := st.chat_input("Ask a question about the uploaded documents:"):
        # Display and record the user's message
        st.session_state.messages.append({"role": "user", "content": user_query})
        with st.chat_message("user"):
            st.markdown(user_query)

        # Run the agent and generate a response
        with st.chat_message("assistant"):
            payload = {"messages": [{"role": "user", "content": user_query}]}
            run_config = {"configurable": {"thread_id": "active_chat_session"}}

            agent_response = st.session_state.agent.invoke(payload, config=run_config)
            output_text = agent_response["messages"][-1].content
            st.markdown(output_text)

        # Store response in session state
        st.session_state.messages.append({"role": "assistant", "content": output_text})

12. Summary and Key Takeaways

Decoupled Agentic Retrieval: Treating retrieval as a tool (@tool) rather than a hardcoded chain allows the LLM to decide dynamically whether to invoke the vector database or answer conversational queries directly.
Directory Ingestion via PyPDFDirectoryLoader: To ingest multiple user-uploaded documents at once, save the raw file buffers to a shared directory and process them in a single batch.
Precision Chunking: Using RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) preserves sentence context across split boundaries without exceeding token limits.
Embedding Alignment: High-dimensional models (such as text-embedding-3-large) provide the semantic resolution needed to match precise domain terms (like medical or legal jargon).
Session State Isolation: Using Streamlit's st.session_state prevents the application from re-indexing documents and re-instantiating agents during normal UI re-renders.
LangGraph Checkpoint Threads: Stateful execution via InMemorySaver requires passing {"configurable": {"thread_id": "<id>"}} to .invoke(), ensuring multi-turn context is tracked reliably.
Message History Management: Manually appending conversation turns to an st.session_state.messages list guarantees that chat bubbles remain visible across UI re-runs.

#25 End-to-End RAG Chatbot with PDF Upload | Streamlit + LangChain + Agentic RAG #rag #aiagents

End-to-End RAG Chatbot with PDF Upload | Streamlit + LangChain + Agentic RAG

1. Architectural Overview: From Standard RAG to Agentic RAG

Key Differences

2. Environment Setup and Library Dependencies

Essential Library Roles

3. Naive CLI Implementation: Ingestion and Retrieval Pipeline

Step 1: Loading and Splitting Documents

Step 2: Vector Embedding and Indexing

4. Building the Retrieval Tool and Agent Reasoning Loop

Similarity Search Mechanics

5. Integrating Conversation Memory via LangGraph Checkpointers

Implementing Agent Memory

6. Runtime Debugging: Real-World Failure Modes and Resolutions

1. The Checkpointer Configuration Error

2. Semantic Misalignment & Embedding Mismatches

3. Callable Graph Runtime Exceptions

7. Transitioning to a Production-Ready Streamlit Interface

8. State Management: Handling Script Re-runs with st.session_state

Session State Variable Schema

9. Dynamic Multi-PDF Document Ingestion Pipeline

Ingestion and Agent Construction Code

10. Message History and Chat UI Rendering

11. End-to-End System Walkthrough

12. Summary and Key Takeaways

8. State Management: Handling Script Re-runs with `st.session_state`