Build Multi-Agent AI System From Scratch | AI Engineer Series

This module covers the practical implementation of Multi-Agent AI Systems from scratch without relying on heavy orchestration frameworks like LangGraph. It explores the transition from a single-agent system to a multi-agent architecture managed by an orchestrator, analyzes token consumption trade-offs, and details why architectural simplicity should always precede complex decentralization.

1. Why Do We Need Multi-Agent Systems? (The Status Quo)

In earlier lessons, single-agent systems were built where a single Large Language Model (LLM) function (ask_llm) received user queries, evaluated requirements, and dynamically decided which tools to invoke (such as Tavily Web Search or a Python calculator).

However, as systems grow in complexity, a single LLM trying to wear multiple hats suffers from reasoning degradation, context pollution, and poor execution guardrails. Multi-agent systems delegate specialized tasks to distinct sub-agents coordinated by a central manager.

Real-Life Analogy: Running a YouTube Channel Alone vs. With a Team

• Single-Agent System: Imagine running an entire YouTube channel completely alone. You research and write the script, record the video, edit the clips, create the thumbnail, and finally upload it to YouTube. If something breaks or a mistake is made during editing, you handle the fix yourself within your own mental workspace.
• Multi-Agent System (Delegation): Now, scale the channel up. You introduce an Operations Manager who sits at the top. The Manager has zero technical skills—they cannot teach, edit, or upload videos. Their sole job is task distribution:
  • When a new video is needed, the Manager hands a task explicitly to the Teacher Agent.
  • Once the raw video returns, the Manager hands it to the Editor Agent.
  • Once editing and thumbnails are finalized, the Manager hands the final package to the Uploader Agent.
  • The Manager maintains a tracking diary to keep note of progress between steps.

2. Core Mental Model: Single-Agent vs. Multi-Agent Architecture

To understand how this translates into code and LLM function calling, we can contrast the two architectural approaches.

Single-Agent Workflow

Multi-Agent Workflow (Manager & Specialized Helpers)

3. Designing the Multi-Agent Components

When constructing the multi-agent layout without LangGraph, three distinct helper and orchestrator components are established using custom Python functions and strict system prompts.

1. The Manager LLM (ask_manager_llm)

• Role: Pure orchestrator and decision-maker.
• Tool Access: None. It cannot search the web or run math calculations.
• Responsibility: Inspects user queries, determines which helper agent to invoke next, tracks responses in a state dictionary, and decides when the task is complete.

2. The Search LLM (ask_search_llm)

• Role: Specialized research assistant.
• Tool Access: Tavily Web Search API exclusively.
• Responsibility: Takes atomic search tasks from the Manager, retrieves real-time data, and returns raw findings without worrying about calculations or final output formatting.

3. The Math LLM (ask_math_llm)

• Role: Specialized calculation engine.
• Tool Access: Python Calculator tool exclusively.
• Responsibility: Performs mathematical operations strictly via code execution, ensuring the LLM never attempts raw mental arithmetic.

4. Implementation Code: Single-Agent vs. Multi-Agent

Below is the implementation pattern requested from the LLM assistant to build and compare both architectures, complete with token usage logging.

Code Snippet: Setting up the Comparison Framework

import os
from openai import OpenAI

# Initialize client (e.g., Groq / OpenAI compatible client)
client = OpenAI(api_key=os.environ.get("GROQ_API_KEY"))

# --- 1. SINGLE AGENT IMPLEMENTATION ---
def ask_single_agent(prompt: str, max_attempts: int = 5):
    """Single LLM instance handling all tools natively."""
    tools = ["web_search", "calculator"]
    # Execution logic where single model decides tool calls dynamically...
    pass

# --- 2. MULTI-AGENT IMPLEMENTATION ---
def ask_manager_llm(task_description: str) -> str:
    """Manager agent with zero tools; routes tasks to helpers."""
    system_prompt = "You are a Manager. You have NO tools. Delegate tasks to Search or Math agents."
    # Orchestration logic...
    pass

def ask_search_llm(query: str) -> str:
    """Specialized agent with access ONLY to web search."""
    system_prompt = "You are a Search Specialist. You ONLY have access to Tavily web search."
    pass

def ask_math_llm(expression: str) -> str:
    """Specialized agent with access ONLY to the calculator."""
    system_prompt = "You are a Math Specialist. You ONLY have access to the python calculator. Never compute in your head."
    pass

5. Real-World Practical Pitfalls & Edge Cases

When deploying multi-agent systems in production, developers immediately encounter several operational hurdles:

• Tool Choice Leakage: Some LLM models ignore tool_choice='none' settings embedded in system prompts. A Manager LLM might mistakenly attempt to invoke a search tool directly despite having no tools attached to its function signature.
• Infinite Loop / Retry Explosions: If a helper agent repeatedly verifies its output or encounters malformed tool payloads, it burns through iteration steps. Setting a strict max_attempts = 5 guardrail prevents runaway infinite loops.
• API Rate Limits & Token Exhaustion: In free tiers or high-volume enterprise pipelines, orchestrating multiple back-and-forth LLM handoffs quickly hits minute-level token quotas (e.g., 8,000 tokens/min), resulting in HTTP 429 Too Many Requests errors.

6. Complete Flow Walkthrough: Complex Query Execution

To test the token consumption and communication overhead, a complex multi-step test query is passed to both architectures:

Step-by-Step Multi-Agent Execution Trace

7. Comparative Evaluation: Token Usage & Cost Analysis

After executing identical complex queries across both systems, token consumption metrics reveal a definitive trade-off:

Crucial Rule of Engineering: If a single-agent system achieves an acceptable accuracy threshold ($\ge 85\%$), do not introduce multi-agent complexity. Only adopt multi-agent architectures when single-agent reasoning fails, or when task domains are radically partitioned (e.g., high-reasoning research vs. low-reasoning boilerplate email generation).

Summary and Key Takeaways

• Definition of Multi-Agent Systems: A hierarchical setup where a tool-less Manager LLM orchestrates specialized helper agents, mimicking real-world corporate management or film production teams.
• Role Isolation: Helper agents are restricted to single domains (e.g., Search-only or Math-only) via targeted system prompts to prevent context pollution and improve execution focus.
• The Communication Overhead: Introducing multiple agents exponentially increases inter-agent communication, leading to higher token consumption and increased infrastructure costs ($\sim 20\% - 40\%$ more tokens).
• The Golden Rule ("If it works, don't touch it"): Avoid blindly building multi-agent systems for simple workflows; always prioritize architectural simplicity until evaluation metrics prove single-agent inadequacy.
• Model Selection Flexibility: Developers can mix and match models across agents—using high-reasoning expensive models (e.g., Claude Opus) for the Manager, and faster, cheaper models (e.g., Claude Sonnet or free tiers) for repetitive helper tasks.
• Guardrails Against Infinite Loops: Always implement strict iteration limits (such as max_attempts = 5) to prevent runaway LLM tool-calling loops and API rate-limit exhaustion.