Building a Self-RAG: Making Agentic Workflows Critique Through Self-Reflection

Retrieval-Augmented Generation (RAG) grounds Large Language Models (LLMs) on external, domain-specific knowledge bases. Standard RAG workflows indiscriminately fetch documents and generate answers, making them susceptible to irrelevant contexts, hallucinations, and unhelpful responses. Self-RAG (Self-Reflective Retrieval-Augmented Generation) addresses these limitations by embedding explicit self-reflection and grading mechanisms directly into the agentic workflow. Using LangChain and LangGraph, this architecture enables LLMs to evaluate document relevance, verify factual grounding (hallucination checks), and grade final answer usefulness before delivering responses.

1. Problem Statement: Limitations of Standard RAG

Standard Retrieval-Augmented Generation follows an uncritical pipeline: it takes a user query, retrieves the top- $k$ vector embeddings from a vector store, passes them into an LLM context window, and returns the generated answer.

Structure & Concept Flow

User Question

Vector Store Retrieval (Top-k Passages)

LLM Context Window (Unfiltered Passages + Query)

Generated Response (Unverified)

Key Failures of the Standard RAG Approach

Indiscriminate Passage Incorporation: Standard RAG retrieves a fixed number of passages regardless of whether retrieval is necessary or whether the passages actually contain relevant information.
Context Window Pollution: Including noise or off-topic context chunks degrades LLM output quality, increases operational costs, and risks exceeding token context limits.
Parametric & Extrapolative Hallucinations: If retrieved passages are partially relevant or contradictory, LLMs tend to fabricate details, relying on internal parametric memory rather than grounded factual facts.
Lack of Critique & Error Recovery: A naive workflow cannot detect when its generated output fails to answer the user's original prompt, returning unhelpful or incorrect answers directly to the user.

2. Research Foundation: Self-RAG & Critique Tokens

The architectural foundation of Self-RAG originates from the research paper "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection" (Asai et al.). The framework introduces explicit reflection tokens (or critique layers) to evaluate quality throughout the retrieval and generation stages.

Self-RAG Reflection Layers

Critique Token / Reflection Layer	Standard Formal Notation	Purpose in Workflow	Target Output / Value
Retrieve / Document Relevancy	`Is-Rel`	Evaluates whether a retrieved passage $D$ contains information relevant to question $X$ .	`[Relevant]` / `[Irrelevant]` (`yes`/`no`)
Is Supported / Hallucination Check	`Is-Sup`	Verifies if the generated statement $Y$ is strictly grounded in and supported by document $D$ .	`[Fully supported]` / `[Has Hallucination]` (`yes`/`no`)
Is Useful / Answer Quality	`Is-Use`	Assesses whether generated response $Y$ provides a helpful, complete resolution to question $X$ .	`[Useful]` (5-point scale or binary `yes`/`no`)

Conceptually: Instead of maintaining an infinite background retry loop—which leads to unpredictable latency, high token consumption, and unbounded API costs—production agentic design introduces deterministic conditional gates. If self-reflection yields negative scores across all candidate context chunks, execution safely terminates or flags the state for human intervention.

3. High-Level Architecture & Workflow Design

The workflow is constructed as a Directed Acyclic Graph (DAG) using LangGraph. Every node represents an isolated step or self-reflection check, while state transitions are governed by conditional edges based on shared state flags.

Structure & Concept Flow

[START]

create_model

build_vector_store

get_relevant_documents

grade_documents (Reflection Layer 1: Is-Rel)

decide_to_generate (Conditional Edge)

/ \

(All docs irrelevant) (Relevant docs exist)

/ \

v v

[END] generate

check_for_hallucination (Reflection Layer 2: Is-Sup)

grade_answer (Reflection Layer 3: Is-Use)

[END]

Architectural Execution Order

create_model: Instantiates and configures the centralized LLM instance (GPT-4o-mini).
build_vector_store: Ingests external URLs, splits documents into numerical embeddings, and stores them in a vector store.
get_relevant_documents: Fetches initial candidate context chunks matching the user's question via vector similarity.
grade_documents: (Self-Reflection Layer 1) Scores each document chunk for binary relevancy (yes/no) against the prompt, filtering out noise.
decide_to_generate: Evaluates the remaining filtered context list. If empty, routes immediately to [END]; otherwise proceed to generation.
generate: Synthesizes a grounded response using the prompt query and filtered documents.
check_for_hallucination: (Self-Reflection Layer 2) Assesses whether the generated response is strictly supported by the retrieved document context.
grade_answer: (Self-Reflection Layer 3) Evaluates whether the generated response directly answers the user's prompt query.

4. State Management & Structured Schema Design

Agentic communication in LangGraph requires a unified Shared State passed sequentially across nodes. Data models derived from Pydantic enforce structured JSON outputs from LLM self-reflection layers.

Shared Graph State (`GraphState`)

The GraphState dictionary acts as the single source of truth for the entire graph:

PYTHON

from typing import List, TypedDict, Any
from langchain_core.documents import Document

class GraphState(TypedDict):
    question: str
    generation: str
    documents: List[Document]
    model: Any
    vector_store: Any
    hallucinated: bool
    valid_answer: bool

Pydantic Data Models for Structured Output

To force the LLM to output consistent evaluation parameters rather than unstructured freeform text, we attach Pydantic models to with_structured_output() calls.

PYTHON

from pydantic import BaseModel, Field

class GradeDocuments(BaseModel):
    """Binary score for document relevancy check."""
    binary_score: str = Field(
        description="Documents are relevant to the question, 'yes' or 'no'"
    )

class GradeHallucinations(BaseModel):
    """Binary score for hallucination evaluation in generated answers."""
    binary_score: str = Field(
        description="Answer is grounded in the facts/documents, 'yes' or 'no'"
    )

class GradeAnswer(BaseModel):
    """Binary score for evaluating whether the answer addresses the question."""
    binary_score: str = Field(
        description="Answer addresses the user question, 'yes' or 'no'"
    )

5. Ingestion Pipeline: Building the Vector Store

Before retrieval occurs, raw source material (e.g., HTML pages, technical newsletters, markdown files, PDFs) must be loaded, chunked, embedded, and indexed inside a high-performance vector database.

Structure & Concept Flow

Web Base Loader (Raw Knowledge Base URLs)

Document Splitting (Recursive Character Text Splitter)

Chunk Size = 250 | Chunk Overlap = 0

OpenAI Embeddings (Text -> Vector Transformation)

Chroma Vector Database (In-Memory Indexing)

Vector Store Implementation Details

Document Loading: WebBaseLoader parses online documents and extracts clear text content.
Text Chunking: RecursiveCharacterTextSplitter breaks down large text documents into discrete tokens. A strict chunk size constraint (e.g., $250$ characters) ensures high semantic density per vector passage.
Embedding Generation & Indexing: OpenAIEmbeddings transforms text chunks into multi-dimensional vectors, indexing them inside an in-memory Chroma database instance.

6. Self-Reflection Layer 1: Document Relevancy Grading

In Layer 1, the model reviews candidate text chunks retrieved by similarity search and strips away irrelevant passages before generation.

PYTHON

# System Prompt for Layer 1: Document Relevancy
DOCUMENT_GRADER_PROMPT = """
You are a grader assessing relevance of a retrieved document to a user question.
It does not need to be a stringent test. The goal is to filter out erroneous retrievals.
If the document contains keywords or semantic meaning related to the user question, grade it as relevant.
Give a binary score 'yes' or 'no' to indicate whether the document is relevant to the question.
"""

Document Filtering Logic

Structure & Concept Flow

Retrieved Vector Chunks [Doc1, Doc2, Doc3]

For Each Document Chunk

LLM Assessment (DOCUMENT_GRADER_PROMPT)

+----------+----------+

| |

Score = 'yes' Score = 'no'

| |

v v

Keep in State Discard Passage

[Filter Docs]

Each chunk undergoes structured evaluation. If the LLM returns binary_score == "yes", the document is saved to filtered_documents. If all documents score "no", filtered_documents returns empty, triggering execution termination downstream.

7. Answer Generation & Dynamic Routing Logic

Conditional Gate: `decide_to_generate`

Once document grading completes, execution passes to a conditional edge router instead of moving blindly to generation.

PYTHON

def decide_to_generate(state: GraphState) -> str:
    """Evaluates presence of relevant documents to route workflow execution."""
    filtered_documents = state["documents"]
    
    if not filtered_documents:
        print("---DECISION: ALL DOCUMENTS ARE IRRELEVANT. TERMINATING WORKFLOW---")
        return "end"
    else:
        print("---DECISION: RELEVANT DOCUMENTS PRESENT. PROCEEDING TO GENERATION---")
        return "continue"

Answer Generation Step

When decide_to_generate returns "continue", the generate node executes. It combines the prompt and filtered context passages into a standard RAG prompt template pulled from the LangChain Hub (rlm/rag-prompt).

\text{Generation} = \text{LLM}(\text{PromptQuery}, \text{FilteredDocuments})

The resulting string generation is stored under state["generation"].

8. Self-Reflection Layer 2: Hallucination Detection

Layer 2 evaluates whether the generated output is grounded exclusively in the provided document facts (Is-Sup).

PYTHON

# System Prompt for Layer 2: Hallucination Evaluation
HALLUCINATION_GRADER_PROMPT = """
You are a grader assessing whether an LLM generation is grounded in or supported by a set of retrieved facts.
Give a binary score 'yes' or 'no'. 'Yes' means that the answer is grounded in or supported by the set of facts.
"""

Evaluation Protocol

Structure & Concept Flow

Filtered Documents + Generated Answer

LLM Critique (HALLUCINATION_GRADER_PROMPT)

+---------+---------+

| |

Score = 'yes' Score = 'no'

| |

v v

No Hallucination Hallucination Detected

hallucinated = False hallucinated = True

If the generated text references external parameters or introduces unsourced metrics, the LLM flags the state by setting state["hallucinated"] = True.

9. Self-Reflection Layer 3: Answer Usefulness Grading

Layer 3 evaluates whether the output resolves the original user query (Is-Use).

PYTHON

# System Prompt for Layer 3: Answer Quality Grading
ANSWER_GRADER_PROMPT = """
You are a grader assessing whether an answer addresses or resolves a question.
Give a binary score 'yes' or 'no'. 'Yes' means that the answer resolves the question.
"""

Evaluation Protocol

Structure & Concept Flow

User Question + Generated Answer

LLM Critique (ANSWER_GRADER_PROMPT)

+---------+---------+

| |

Score = 'yes' Score = 'no'

| |

v v

Valid Answer Invalid Output

valid_answer = True valid_answer = False

If the model produces a generic evasion (e.g., "I am unable to locate that details"), the grader sets state["valid_answer"] = False. This signals to upstream observability systems that the system failed to deliver a useful answer.

10. Graph Assembly & Execution Control Flow

LangGraph manages graph building, node attachments, explicit state link wiring, conditional routing logic, and runtime graph compilation.

Structure & Concept Flow

[START] -> create_model -> build_vector_store -> get_relevant_documents -> grade_documents

decide_to_generate

/ \

("end") ("continue")

/ \

v v

[END] generate

check_for_hallucination

grade_answer

[END]

Construction Pattern

PYTHON

from langgraph.graph import StateGraph, START, END

def build_graph() -> Any:
    workflow = StateGraph(GraphState)

    # 1. Register Graph Nodes
    workflow.add_node("create_model", create_model)
    workflow.add_node("build_vector_store", build_vector_store)
    workflow.add_node("get_relevant_documents", get_relevant_documents)
    workflow.add_node("grade_documents", grade_documents)
    workflow.add_node("generate", generate)
    workflow.add_node("check_for_hallucination", check_for_hallucination)
    workflow.add_node("grade_answer", grade_answer)

    # 2. Wire Direct Sequential Edges
    workflow.add_edge(START, "create_model")
    workflow.add_edge("create_model", "build_vector_store")
    workflow.add_edge("build_vector_store", "get_relevant_documents")
    workflow.add_edge("get_relevant_documents", "grade_documents")

    # 3. Register Conditional Control Edges
    workflow.add_conditional_edges(
        "grade_documents",
        decide_to_generate,
        {
            "continue": "generate",
            "end": END,
        },
    )

    # 4. Wire Downstream Self-Reflection Edges
    workflow.add_edge("generate", "check_for_hallucination")
    workflow.add_edge("check_for_hallucination", "grade_answer")
    workflow.add_edge("grade_answer", END)

    # 5. Compile & Return Executable Graph
    return workflow.compile()

11. End-to-End Code Walkthrough

Below is the complete, single-file implementation of the Self-RAG agentic system using LangChain, LangGraph, ChromaDB, and OpenAI models.

PYTHON

import os
from typing import List, TypedDict, Any
from pydantic import BaseModel, Field

from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain import hub
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langgraph.graph import StateGraph, START, END

# ==========================================
# 1. PROMPT DEFINITIONS
# ==========================================

DOCUMENT_GRADER_PROMPT = """
You are a grader assessing relevance of a retrieved document to a user question.
It does not need to be a stringent test. The goal is to filter out erroneous retrievals.
If the document contains keywords or semantic meaning related to the user question, grade it as relevant.
Give a binary score 'yes' or 'no' to indicate whether the document is relevant to the question.
"""

HALLUCINATION_GRADER_PROMPT = """
You are a grader assessing whether an LLM generation is grounded in or supported by a set of retrieved facts.
Give a binary score 'yes' or 'no'. 'Yes' means that the answer is grounded in or supported by the set of facts.
"""

ANSWER_GRADER_PROMPT = """
You are a grader assessing whether an answer addresses or resolves a question.
Give a binary score 'yes' or 'no'. 'Yes' means that the answer resolves the question.
"""

# ==========================================
# 2. SHARED STATE & DATA MODELS
# ==========================================

class GraphState(TypedDict):
    question: str
    generation: str
    documents: List[Any]
    model: Any
    vector_store: Any
    hallucinated: bool
    valid_answer: bool

class GradeDocuments(BaseModel):
    binary_score: str = Field(description="Documents are relevant to question, 'yes' or 'no'")

class GradeHallucinations(BaseModel):
    binary_score: str = Field(description="Answer is grounded in facts, 'yes' or 'no'")

class GradeAnswer(BaseModel):
    binary_score: str = Field(description="Answer addresses the question, 'yes' or 'no'")

# Knowledge base source URLs
KNOWLEDGE_BASE_URLS = [
    "https://newsletter.systemsthatscale.org/p/parallel-execution-of-nodes-in-langgraph",
    "https://newsletter.systemsthatscale.org/p/tool-calling-in-langchain",
]

# ==========================================
# 3. GRAPH NODES IMPLEMENTATION
# ==========================================

def create_model(state: GraphState) -> GraphState:
    """Instantiates centralized LLM configuration."""
    print("---NODE: CREATING LLM MODEL---")
    model = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    state["model"] = model
    return state

def build_vector_store(state: GraphState) -> GraphState:
    """Loads external web sources and builds an in-memory Chroma vector database."""
    print("---NODE: BUILDING VECTOR STORE---")
    docs = [WebBaseLoader(url).load() for url in KNOWLEDGE_BASE_URLS]
    docs_list = [item for sublist in docs for item in sublist]

    text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
        chunk_size=250, chunk_overlap=0
    )
    doc_splits = text_splitter.split_documents(docs_list)

    vector_store = Chroma.from_documents(
        documents=doc_splits,
        embedding=OpenAIEmbeddings(),
        collection_name="self-rag-chroma",
    )
    state["vector_store"] = vector_store
    return state

def get_relevant_documents(state: GraphState) -> GraphState:
    """Retrieves context documents matching user prompt vector distance."""
    print("---NODE: RETRIEVING RELEVANT DOCUMENTS---")
    question = state["question"]
    vector_store = state["vector_store"]
    
    documents = vector_store.similarity_search(question)
    state["documents"] = documents
    return state

def grade_documents(state: GraphState) -> GraphState:
    """Layer 1: Filters retrieved documents based on binary relevancy to question."""
    print("---NODE: REFLECTION LAYER 1 (GRADE DOCUMENTS)---")
    question = state["question"]
    documents = state["documents"]
    model = state["model"]

    structured_llm_grader = model.with_structured_output(GradeDocuments)
    grade_prompt = ChatPromptTemplate.from_messages([
        ("system", DOCUMENT_GRADER_PROMPT),
        ("human", "Retrieved document:\n\n{document}\n\nUser question: {question}"),
    ])
    retrieval_grader = grade_prompt | structured_llm_grader

    filtered_documents = []
    for doc in documents:
        score = retrieval_grader.invoke({"question": question, "document": doc.page_content})
        if score.binary_score.lower() == "yes":
            print("  -> EVALUATION: Document passage relevant.")
            filtered_documents.append(doc)
        else:
            print("  -> EVALUATION: Document passage irrelevant. Dropping.")

    state["documents"] = filtered_documents
    return state

def decide_to_generate(state: GraphState) -> str:
    """Conditional Edge Gate: Evaluates document state to proceed or abort."""
    print("---CONDITIONAL EDGE: DECIDING TO GENERATE---")
    if not state["documents"]:
        print("  -> DECISION: All documents irrelevant. Stopping graph execution.")
        return "end"
    print("  -> DECISION: Relevant context chunks verified. Proceeding to generation.")
    return "continue"

def generate(state: GraphState) -> GraphState:
    """Generates an answer using the filtered context documents."""
    print("---NODE: GENERATING RESPONSE---")
    question = state["question"]
    documents = state["documents"]
    model = state["model"]

    prompt = hub.pull("rlm/rag-prompt")
    rag_chain = prompt | model | StrOutputParser()

    generation = rag_chain.invoke({"context": documents, "question": question})
    state["generation"] = generation
    return state

def check_for_hallucination(state: GraphState) -> GraphState:
    """Layer 2: Evaluates if generated text is grounded in document facts."""
    print("---NODE: REFLECTION LAYER 2 (HALLUCINATION CHECK)---")
    documents = state["documents"]
    generation = state["generation"]
    model = state["model"]

    structured_llm_grader = model.with_structured_output(GradeHallucinations)
    hallucination_prompt = ChatPromptTemplate.from_messages([
        ("system", HALLUCINATION_GRADER_PROMPT),
        ("human", "Set of facts:\n\n{documents}\n\nLLM generation: {generation}"),
    ])
    grader = hallucination_prompt | structured_llm_grader

    score = grader.invoke({"documents": documents, "generation": generation})
    if score.binary_score.lower() == "yes":
        print("  -> EVALUATION: Answer is fully grounded in retrieved context.")
        state["hallucinated"] = False
    else:
        print("  -> EVALUATION: Hallucination detected! Output not grounded.")
        state["hallucinated"] = True

    return state

def grade_answer(state: GraphState) -> GraphState:
    """Layer 3: Evaluates if generated text directly resolves user prompt query."""
    print("---NODE: REFLECTION LAYER 3 (ANSWER RELEVANCY)---")
    question = state["question"]
    generation = state["generation"]
    model = state["model"]

    structured_llm_grader = model.with_structured_output(GradeAnswer)
    answer_prompt = ChatPromptTemplate.from_messages([
        ("system", ANSWER_GRADER_PROMPT),
        ("human", "User question:\n\n{question}\n\nLLM generation: {generation}"),
    ])
    grader = answer_prompt | structured_llm_grader

    score = grader.invoke({"question": question, "generation": generation})
    if score.binary_score.lower() == "yes":
        print("  -> EVALUATION: Generated answer effectively resolves user question.")
        state["valid_answer"] = True
    else:
        print("  -> EVALUATION: Generated answer fails to resolve question.")
        state["valid_answer"] = False

    return state

# ==========================================
# 4. GRAPH CONSTRUCTION
# ==========================================

def build_graph() -> Any:
    workflow = StateGraph(GraphState)

    workflow.add_node("create_model", create_model)
    workflow.add_node("build_vector_store", build_vector_store)
    workflow.add_node("get_relevant_documents", get_relevant_documents)
    workflow.add_node("grade_documents", grade_documents)
    workflow.add_node("generate", generate)
    workflow.add_node("check_for_hallucination", check_for_hallucination)
    workflow.add_node("grade_answer", grade_answer)

    workflow.add_edge(START, "create_model")
    workflow.add_edge("create_model", "build_vector_store")
    workflow.add_edge("build_vector_store", "get_relevant_documents")
    workflow.add_edge("get_relevant_documents", "grade_documents")

    workflow.add_conditional_edges(
        "grade_documents",
        decide_to_generate,
        {
            "continue": "generate",
            "end": END,
        },
    )

    workflow.add_edge("generate", "check_for_hallucination")
    workflow.add_edge("check_for_hallucination", "grade_answer")
    workflow.add_edge("grade_answer", END)

    return workflow.compile()

12. Real-World Execution & Edge-Case Validation

To observe the Self-RAG framework in practice, we test both an in-domain relevant query and an out-of-domain irrelevant query.

Test Case A: In-Domain Prompt Query

PYTHON

# Execution Driver Initialization
if __name__ == "__main__":
    app = build_graph()
    
    # In-Domain Query matching ingested knowledge base topic
    query = {"question": "What is tool calling?"}
    print(f"\n================ EXECUTION RUN A: '{query['question']}' ================\n")
    response = app.invoke(query)
    
    # State Inspection Safety Guards
    if "hallucinated" in response:
        if response["hallucinated"]:
            print("\n[WARNING]: State flags indicate output hallucination!")
    if "valid_answer" in response:
        if not response["valid_answer"]:
            print("\n[WARNING]: State flags indicate output fails query resolution!")
            
    if "generation" in response:
        print(f"\nFINAL SYSTEM RESPONSE:\n{response['generation']}")

Execution Terminal Log A

Structure & Concept Flow

================ EXECUTION RUN A: 'What is tool calling?' ================

---NODE: CREATING LLM MODEL---

---NODE: BUILDING VECTOR STORE---

---NODE: RETRIEVING RELEVANT DOCUMENTS---

---NODE: REFLECTION LAYER 1 (GRADE DOCUMENTS)---

-> EVALUATION: Document passage relevant.

---CONDITIONAL EDGE: DECIDING TO GENERATE---

-> DECISION: Relevant context chunks verified. Proceeding to generation.

---NODE: GENERATING RESPONSE---

---NODE: REFLECTION LAYER 2 (HALLUCINATION CHECK)---

-> EVALUATION: Answer is fully grounded in retrieved context.

---NODE: REFLECTION LAYER 3 (ANSWER RELEVANCY)---

-> EVALUATION: Generated answer effectively resolves user question.

FINAL SYSTEM RESPONSE:

Tool calling is a concept in LangChain where large language models interpret human messages and determine which external tool to invoke based on context. These tools can include APIs, custom functions, or databases, allowing the AI model to perform specific tasks with appropriate parameters.

Test Case B: Out-of-Domain / Irrelevant Query

Now we test an off-topic query completely absent from the ingested vector store documents:

PYTHON

query_b = {"question": "What is a disjoint set data structure?"}
print(f"\n================ EXECUTION RUN B: '{query_b['question']}' ================\n")
response_b = app.invoke(query_b)

Execution Terminal Log B

Structure & Concept Flow

================ EXECUTION RUN B: 'What is a disjoint set data structure?' ================

---NODE: CREATING LLM MODEL---

---NODE: BUILDING VECTOR STORE---

---NODE: RETRIEVING RELEVANT DOCUMENTS---

---NODE: REFLECTION LAYER 1 (GRADE DOCUMENTS)---

-> EVALUATION: Document passage irrelevant. Dropping.

---CONDITIONAL EDGE: DECIDING TO GENERATE---

-> DECISION: All documents irrelevant. Stopping graph execution.

[INFO]: Workflow safely terminated at conditional gate. Generation bypassed.

Analysis of Test Case B Results

Self-Correction Interception: Reflection Layer 1 (grade_documents) analyzed all retrieved candidates, identified zero keyword or semantic overlap, and stripped all documents from state.
Resource & Cost Optimization: decide_to_generate routed execution directly to END. The system avoided invoking answer generation, hallucination checks, and usefulness grading—saving API tokens and execution time.
Prevention of Hallucination: Bypassing generation prevented the LLM from synthesizing an ungrounded or hallucinated answer when factual context was missing.

13. Summary and Key Takeaways

Standard RAG Bottlenecks: Unfiltered retrieval leads to context window pollution, model hallucinations, and unhelpful responses.
The Self-RAG Model: Derived from research by Asai et al., Self-RAG adds critique layers (Is-Rel, Is-Sup, Is-Use) directly into the agent workflow.
Structured Self-Reflection: Combining Pydantic schema constraints with LangChain structured output forcing (with_structured_output) ensures consistent binary evaluations (yes/no).
Tri-Layer Reflection Strategy:
1. Layer 1 (Document Relevancy): Filters out non-matching vector contexts before generation.
2. Layer 2 (Hallucination Detection): Verifies that generated claims are fully supported by context facts.
3. Layer 3 (Answer Quality Grading): Confirms that the final output directly answers the user's prompt query.
Deterministic Routing: Conditional edges route state execution dynamically—halting execution early if contexts are missing, preventing ungrounded generations, and protecting operational token budgets.

Building a Self-RAG | Making your Agentic workflows critique through Self-Reflection | LangChain

Building a Self-RAG: Making Agentic Workflows Critique Through Self-Reflection

1. Problem Statement: Limitations of Standard RAG

Key Failures of the Standard RAG Approach

2. Research Foundation: Self-RAG & Critique Tokens

Self-RAG Reflection Layers

3. High-Level Architecture & Workflow Design

Architectural Execution Order

4. State Management & Structured Schema Design

Shared Graph State (GraphState)

Pydantic Data Models for Structured Output

5. Ingestion Pipeline: Building the Vector Store

Vector Store Implementation Details

6. Self-Reflection Layer 1: Document Relevancy Grading

Document Filtering Logic

7. Answer Generation & Dynamic Routing Logic

Conditional Gate: decide_to_generate

Answer Generation Step

8. Self-Reflection Layer 2: Hallucination Detection

Evaluation Protocol

9. Self-Reflection Layer 3: Answer Usefulness Grading

Evaluation Protocol

10. Graph Assembly & Execution Control Flow

Construction Pattern

11. End-to-End Code Walkthrough

12. Real-World Execution & Edge-Case Validation

Test Case A: In-Domain Prompt Query

Execution Terminal Log A

Test Case B: Out-of-Domain / Irrelevant Query

Execution Terminal Log B

Analysis of Test Case B Results

13. Summary and Key Takeaways

Shared Graph State (`GraphState`)

Conditional Gate: `decide_to_generate`