Building a Deep Research Agent: Static Hierarchical Workflows and Benchmarking

This study note dissects the architecture, implementation, and benchmarking of a configurable, static deep research agent based on the paper “A Hierarchical Tree-Based Approach for Creating Configurable and Static Deep Research Agent”. Grounded in Gemini 2.5 Pro and validated on DeepResearchBench, this chapter explores how deterministic hierarchical workflows, controlled parameter scaling, and strict search filtering yield high-density research reports while containing token costs.

1. Static vs. Dynamic Agentic Architectures

When designing autonomous LLM-powered agents for complex tasks like multi-step academic or market research, system architects face a fundamental choice between static and dynamic workflows.

Conceptually: A static workflow acts like an assembly line with strict quality checkpoints and defined worker roles. The agent is strictly constrained to run down pre-engineered structural branches rather than inventing new tool loops on the fly.

2. Core Hierarchical Mental Model: Tree Decomposition and Recurrent Backtracking

The core operational model of this Deep Research Agent (DRA) treats a research problem as a tree traversal problem. A broad input query acts as the root node, which undergoes recursive subtopic decomposition until it yields atomic leaf queries that can be answered independently.

Real-Life Analogy: Executive Research Delegation

Think of a Chief Research Officer (Root Node) assigned to draft a comprehensive dossier on global investment strategies. Instead of diving into micro-details directly, the officer divides the task among Senior Directors (Subtopic Nodes: Warren Buffett, Charlie Munger, Duan Yongping).

Each Senior Director splits their topic further into specific sub-tenets (e.g., Circle of Competence). Junior Researchers (Worker Nodes / Leaf Topics) receive narrow, single-topic assignments, perform strict library/web queries, and draft targeted briefs. The briefs are passed back up the management chain, where Directors compile them into cohesive chapters before presenting the consolidated report to the Chief Officer.

3. Structural Parameters: Depth ($D$) and Breadth ($B$) Mechanics

The Deep Research Agent's tree expansion is governed by two structural hyper-parameters: Depth ($D$) and Breadth ($B$).

• Depth ($D$): Specifies how many hierarchical levels down the agent can drill into a given topic. $D$ decrements by $1$ at each successive subtopic layer ($D \rightarrow D - 1$). When $D = 0$, the node is classified as a mandatory leaf topic and sent directly to a worker agent.
• Breadth ($B$): Specifies the maximum number of subtopics that a single parent topic can be split into ($B_{\text{max}} = B$).

Bounded Ceiling Behavior

Breadth $B$ functions strictly as an upper bound (ceiling) rather than a rigid requirement. If an independent agent determines that a query naturally breaks into fewer subtopics than $B$, it will generate only the logically necessary count.

Dynamic Depth and Breadth Reduction Strategy

Human researchers spend significant time structuring high-level core topics, but narrow down sub-branches rapidly to avoid irrelevant rabbit holes. To mimic this behavior, the Deep Research Agent applies a dynamic parameter reduction heuristic as it descends the tree:

1. Depth Reduction: Decrements monotonically: $D_{\text{child}} = D_{\text{parent}} - 1$.
2. Breadth Reduction: Reduces exponentially or linearly as depth decreases: $B_{\text{child}} = \max\left(1, \left\lfloor \frac{B_{\text{parent}}}{2} \right\rfloor\right)$.

This prevents combinatorial explosion at lower tree levels while preserving broad exploration at the top level.

4. Mathematical Complexity and Token Cost Explosion

Unbounded recursive agents suffer from exponential state expansion. Understanding the upper bound of generated queries is essential for budgeting token consumption and execution latency.

Uniform Tree Decomposition Upper Bound

For a uniform tree structure with constant depth $D$ and constant breadth $B$, the total number of research leaf nodes $N_{\text{leaf}}$ and total nodes $N_{\text{total}}$ generated is given by:

Thus, total model interactions grow with asymptotic time complexity:

Warning: Pushing $D$ or $B$ past moderate thresholds (e.g., $D > 3, B > 5$) triggers exponential token generation, potentially incurring high API costs per report with marginal gains in research quality.

5. Agent Role Specialization: Supervisor, Independent, and Worker Agents

The architecture decouples responsibilities across three distinct agent types to isolate decision-making, task decomposition, and execution.

1. Supervisor Agent

• Role: Orchestrator and state evaluator.
• Function: Receives a node context, checks if the remaining depth $D > 0$, and evaluates whether further sub-decomposition is required via a boolean can_split flag.

2. Independent Agent

• Role: Structural analyst.
• Function: Triggered when can_split == True. Analyzes the prompt and generates a list of up to $B$ discrete sub-queries. It reduces parameters ($D - 1$, $B / 2$) before passing control back to the supervisor.

3. Worker Agent

• Role: Execution engine (Leaf Node).
• Function: Executed when can_split == False or $D = 0$. Calls external web search APIs, aggregates context, extracts references, and produces a raw markdown research section.

6. Information Retrieval and Threshold Filtering Strategy

To maintain factual density and minimize hallucination, Worker Agents integrate real-time web retrieval powered by the Tavily Search API.

Web Search Relevance Threshold

Web search returns vary in quality. To prevent noise propagation into the synthesized output, the system enforces a strict Relevance Score Threshold ($\tau = 0.30$):

• Noise Filtering: Results scoring below $30\%$ relevance are purged prior to passing the context window to Gemini 2.5 Pro.
• Citation Preservation: Source URLs from retained documents are preserved to populate an end-of-report bibliography.

7. Performance Benchmarking: DeepResearchBench, RACE, and FACT Frameworks

To establish empirical rigor, the Deep Research Agent was benchmarked using DeepResearchBench, a standard test suite composed of 100 PhD-level research questions (50 English, 50 Chinese) across 20 domain disciplines (Finance, Science, Technology, Social Sciences, etc.).

Evaluation Framework Breakdown

1. RACE Framework: Evaluates general content quality across five dimensions:
   • Readability: Structural clarity, formatting, logical flow.
   • Acuracy / Depth: Granularity of analytical insight.
   • Comprehensiveness: Coverage of the topic space.
   • Execution / Instruction Following: Adherence to user prompt constraints.
2. FACT Framework: Evaluates claims against ground-truth corpora to verify factual precision.

Benchmark Results (Gemini 2.5 Pro Base Engine)

8. Empirical Cost-Performance Trade-offs and Score Plateauing

Running experiments across various parameter pairs ($D, B$) reveals crucial insights into output volume, evaluation performance, and cost bounds.

Configuration Performance Comparison

Critical Pedagogical Takeaways:

1. Diminishing Returns (The Score Plateau): Moving from $D2B3$ to $D2B5$ increases the generated text volume by $6.46\times$ (11.53 KB to 74.50 KB) and query expansion by $4.33\times$, but yields only a modest score increase from $0.35$ to $0.41$.
2. Optimal Efficiency Point: Configuration $D2B3$ represents the optimal trade-off point between LLM token expenses, report concise readability, and benchmark accuracy.
3. Readability Penalty: Extremely deep structures ($D \ge 3, B \ge 5$) generate excessively long reports that degrade cognitive utility for human readers while exponentially inflating API charges.

9. Code Evolution: Implementing the Static Deep Research Agent in Python

The complete execution model of the Static Deep Research Agent is contained within a single Python script (deep_research_agent.py, ~374 lines). Below is a clean implementation demonstrating state handling, supervisor decision loops, Tavily filtering, and subtopic execution.

import os
import time
from typing import List, Dict, Any, Optional
from dataclasses import dataclass, field
from tavily import TavilyClient
import google.generativeai as genai

# --- Configuration & Initialization ---
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")

tavily_client = TavilyClient(api_key=TAVILY_API_KEY)
genai.configure(api_key=GEMINI_API_KEY)
model = genai.GenerativeModel("gemini-2.5-pro")

RELEVANCE_THRESHOLD = 0.30  # Filter search results under 30% relevance

@dataclass
class ResearchState:
    task_id: str
    query: str
    depth: int
    breadth: int
    can_split: bool = True
    parent_path: List[str] = field(default_factory=list)

def tavily_search_filtered(query: str) -> List[Dict[str, Any]]:
    """Executes search and applies relevance threshold filtering."""
    response = tavily_client.search(query=query, max_results=5)
    results = response.get("results", [])
    
    filtered_results = [
        res for res in results 
        if res.get("score", 0.0) >= RELEVANCE_THRESHOLD
    ]
    return filtered_results

def supervisor_agent(state: ResearchState) -> bool:
    """Determines whether a given query node should be split further."""
    if state.depth <= 0:
        return False
    
    prompt = f"""
    You are a Research Supervisor. Given the research query: "{state.query}".
    Decide if this topic is complex enough to require breakdown into subtopics.
    Respond strictly with 'YES' or 'NO'.
    """
    response = model.generate_content(prompt).text.strip().upper()
    return "YES" in response

def independent_agent_split(state: ResearchState) -> List[str]:
    """Splits a topic into at most 'breadth' subtopics."""
    prompt = f"""
    You are an Independent Research Analyst. Decompose the following topic into 
    at most {state.breadth} independent, high-impact subtopics.
    
    Topic: "{state.query}"
    Provide output as a simple numbered list, one subtopic per line.
    """
    response = model.generate_content(prompt).text.strip()
    subtopics = [
        line.strip() for line in response.split("\n") 
        if line.strip() and line[0].isdigit()
    ]
    return subtopics[:state.breadth]

def worker_agent_research(query: str) -> Dict[str, Any]:
    """Performs web search retrieval, context filtering, and section generation."""
    search_data = tavily_search_filtered(query)
    
    context_str = ""
    citations = []
    for item in search_data:
        context_str += f"- {item['content']} (Source: {item['url']})\n"
        citations.append(item['url'])
        
    prompt = f"""
    You are an expert Research Worker. Write a detailed analysis for the topic: "{query}".
    Use the provided web search context to ground your report. Include inline links.
    
    Context:
    {context_str}
    
    Write a clear, structured Markdown section.
    """
    report_section = model.generate_content(prompt).text.strip()
    return {"content": report_section, "citations": citations}

def execute_deep_research(state: ResearchState) -> Dict[str, Any]:
    """Main static workflow state machine execution logic."""
    time.sleep(1)  # Prevent rate limit violations
    
    should_split = supervisor_agent(state)
    
    if should_split and state.depth > 0:
        subtopics = independent_agent_split(state)
        accumulated_reports = []
        all_citations = []
        
        # Calculate reduced parameter bounds for child nodes
        next_depth = state.depth - 1
        next_breadth = max(1, state.breadth // 2)
        
        for sub in subtopics:
            child_state = ResearchState(
                task_id=state.task_id,
                query=sub,
                depth=next_depth,
                breadth=next_breadth,
                parent_path=state.parent_path + [state.query]
            )
            child_res = execute_deep_research(child_state)
            accumulated_reports.append(child_res["content"])
            all_citations.extend(child_res["citations"])
            
        combined_content = f"## {state.query}\n\n" + "\n\n".join(accumulated_reports)
        return {"content": combined_content, "citations": list(set(all_citations))}
    else:
        # Atomic leaf node execution
        return worker_agent_research(state.query)

10. Execution Walkthrough and Output Structure

Command Line Invocation

To execute the agent locally on a machine, set environment variables and call the script directly:

export GEMINI_API_KEY="your-gemini-key"
export TAVILY_API_KEY="your-tavily-key"

python deep_research_agent.py

Trace Log Output

Generated Markdown Report Topology

The agent structures all responses deterministically, compiling nested sections into a formatted document:

# Deep Research Report: Investment Philosophies

## Table of Contents
1. Core Tenets of Warren Buffett's Value Investing
   - Circle of Competence and Economic Moats
2. Duan Yongping's Modern Capital Allocation Framework
3. Charlie Munger's Mental Models and Multidisciplinary Thinking
4. Works Cited & References

---

## 1. Core Tenets of Warren Buffett's Value Investing
Warren Buffett's framework relies primarily on business clarity and durable competitive advantage...

### Circle of Competence and Economic Moats
Understanding one's operational boundary prevents capital misallocation [Read More](https://example.com/buffett-moats)...

---

## Works Cited & References
- https://example.com/buffett-moats
- https://example.com/duan-yongping-notes

11. Real-World Engineering Pitfalls and Production Considerations

1. Rate Limiting & Throttling:
   • Issue: Parallel leaf node execution can hit LLM API rate limits (RPM/TPM) and Tavily search concurrency limits.
   • Mitigation: Introduce artificial delays (time.sleep(1)), token bucket algorithms, or backoff retry wrappers (tenacity library) around worker nodes.
2. Context Window Overflows during Accumulation:
   • Issue: As partial worker briefs bubble back up to the root, accumulating raw text into higher-level supervisors can exceed context limits.
   • Mitigation: Implement a Summarize-on-Backtrack strategy where supervisors aggregate child briefs into distilled summaries before returning up the stack.
3. Noise Accumulation from Web Search:
   • Issue: Low-quality blogs and redundant SEO pages pollute LLM generation context.
   • Mitigation: Enforce the $30\%$ minimum relevance threshold score ($\tau = 0.30$) and restrict domain lists to reputable source indexes.
4. Static Trajectory Rigidness:
   • Issue: Static workflows cannot adapt if a key sub-branch encounters a dead end.
   • Mitigation: Use static structures for core report frameworks, but allow localized dynamic fallback branches within individual worker nodes.

Summary and Key Takeaways

• Static vs. Dynamic Paradigm: Static deep research agents rely on hardcoded structural pipelines, trading arbitrary runtime tool flexibility for high predictability, debugging traceably, and strict cost containment.
• Hierarchical Tree Structural Processing: Research tasks expand as tree nodes where Supervisor Agents evaluate splitting feasibility, Independent Agents bound breadth ($B$), and Worker Agents execute atomic search tasks at leaf nodes ($D = 0$).
• Ceiling Bound Mechanics: The breadth parameter ($B$) acts as an upper limit, allowing independent agents to spawn fewer subtopics if logical constraints demand it.
• Dynamic Breadth Reduction: Decreasing $B$ dynamically as tree depth $D$ deepens mirrors human research priorities, keeping lower-level subtopic queries targeted and preventing token bloat.
• Exponential Cost Bounds: Unbounded tree expansion grows at $\mathcal{O}(B^D)$. Cost management requires explicitly bounding both $D$ and $B$.
• Strict Quality Control via Thresholding: Tavily search integration mandates a relevance score threshold ($\ge 30\%$) to discard noise, ensuring factual context before text synthesis.
• The Score Plateau Phenomenon: Benchmark results on DeepResearchBench show that moving from $D2B3$ to $D2B5$ increases report size by over $6\times$ while yielding only marginal score improvements ($0.35 \rightarrow 0.41$).
• Optimal Production Configuration: Depth $2$ and Breadth $3$ ($D2B3$) provides the ideal trade-off point between token cost, execution time, readable report density, and RACE benchmark accuracy.