Our First AI Agent: Building Autonomous Tool-Using Agents from Scratch
This chapter introduces the fundamental mechanics of building an AI Agent from first principles, without relying on complex orchestration frameworks like LangGraph. By augmenting a Large Language Model (LLM) with deterministic Python functions and external APIs, you will learn how models reason about user queries, select appropriate tools, and autonomously complete multi-step tasks.
1. Fundamentals: What Defines an AI Agent?
At its core, a standalone Large Language Model (LLM) is a text-in, text-out pattern recognition engine. It predicts the next most probable token based on its training data. However, a raw LLM cannot execute code, retrieve live state from the internet, or interact with databases on its own.
An AI Agent is created when an LLM is paired with external capabilities—referred to in agentic systems as Tools (functions or APIs).
Structure & Concept Flow
Real-Life Analogy: The API Customer Service Representative
- Paytm Flight Booking: Imagine Paytm sets up a dedicated customer service representative at a desk. If you ask this representative for flight schedules from Bengaluru to Delhi, they query Paytm's internal database and return live options and pricing.
- BookMyShow Ticket Booking: To book a ticket for a movie (e.g., Toxic), ChatGPT cannot directly print seats or charge a credit card. It needs an interface to BookMyShow's service representative (an API) who handles seat availability and payment links.
- The LLM as the Decision Maker: The LLM acts as the human user's intelligent interface, while the API endpoint acts as the real-world actor. Connecting the LLM to these APIs transforms a basic conversational bot into an Autonomous Agent.
2. Why LLMs Need External Tools
Despite their vast contextual capabilities, raw LLMs suffer from two major operational limitations:
- Inability to Perform Exact Mathematical Reasoning: LLMs generate text stochastically (probability-based). When given complex arithmetic like or long numeric expressions, they often hallucinate numbers rather than calculate them. They require access to a deterministic code execution environment (like a Python interpreter or calculator).
- Static Knowledge Cutoffs: An LLM's knowledge snapshot stops at the end of its training run (e.g., September 3). It cannot natively answer questions about real-time events, current pricing, or volatile facts (e.g., today's flight prices or live news) without querying external search APIs.
Note: When an LLM determines that its static knowledge or internal capabilities are insufficient, it halts generation and emits a structured Tool Call request back to your application environment.
3. Designing a Dual-Tool AI Agent Architecture
To understand how agents operate under the hood, we design a lightweight agent equipped with two distinct tools:
- Math Calculator Tool: Evaluates mathematical expressions using a Python execution engine.
- Web Search Tool: Fetches live internet data using the Tavily API (a specialized search API tailored for LLMs).
Structure & Concept Flow
4. The OpenAI Tool Specification Schema
For an LLM to decide when and how to invoke a tool, it must be provided with a formal JSON schema specifying the tool's interface, expected arguments, and clear usage descriptions.
Breakdown of Schema Elements
Tool Schema Example (JSON Definition)
JSON
[
{
"type": "function",
"function": {
"name": "web_search",
"description": "Search the web for up-to-date information, current affairs, changing facts, or live pricing.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The concise search query string to execute on the web."
}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate",
"description": "Perform mathematical calculations for arithmetic queries.",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Mathematical expression string, e.g., '10 * 21' or '3 * 90'."
}
},
"required": ["expression"]
}
}
}
]
5. Step-by-Step Execution Flow & Iteration Control
Executing an agentic tool call requires a loop where the LLM can issue requests, receive local runtime responses, and then synthesize a final answer.
Vertical Execution Workflow
Structure & Concept Flow
Iteration Boundaries (Safeguarding Loops)
Complex or ambiguous prompts can occasionally cause an agent to invoke tools endlessly. To protect your API budget and avoid infinite execution loops, an explicit Max Iterations Limit (e.g., ) must be enforced in code.
Structure & Concept Flow
6. Live Code Architecture: Prompting & Implementation
When instructing coding LLMs (such as Claude or ChatGPT) to write complete agent boilerplate, provide explicit functional specifications.
Meta-Prompting Strategy for Code Generation
TEXT
I am making an AI agent using Grok LLM that can do two things:
1. Web Search using the Tavily API (key stored in .env as TAVILY_API_KEY).
2. Mathematical Calculations using a local Python evaluator.
Requirements:
- Use grok/openAI compatible client.
- Model: 'lama-3.3-70b-versatile' or valid Grok endpoint.
- Define two tool definitions using the standard OpenAI JSON schema format (type, name, description, parameters).
- Set tool_choice="auto".
- Implement an agent function with an iteration limit (max 3 loops).
- Write clean Python code utilizing python-dotenv.
Complete Working Python Agent Implementation (my_agent.py)
PYTHON
import os
import ast
from dotenv import load_dotenv
from openai import OpenAI
from tavily import TavilyClient
# 1. Environment Configuration
load_dotenv()
GROK_API_KEY = os.getenv("GROK_API_KEY")
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
# Initialize API clients
client = OpenAI(
api_key=GROK_API_KEY,
base_url="https://api.x.ai/v1" # Grok API Endpoint
)
tavily_client = TavilyClient(api_key=TAVILY_API_KEY)
# 2. Local Tool Implementations
def web_search(query: str) -> str:
"""Executes live web search query using Tavily API."""
try:
response = tavily_client.search(query=query, max_results=3)
results = [res["content"] for res in response.get("results", [])]
return "\n\n".join(results) if results else "No relevant web results found."
except Exception as e:
return f"Error executing web search: {str(e)}"
def calculate(expression: str) -> str:
"""Evaluates mathematical expressions safely using AST parsing."""
try:
# Safely evaluate mathematical expressions
node = ast.parse(expression, mode='eval')
result = eval(compile(node, '<string>', 'eval'), {"__builtins__": None}, {})
return str(result)
except Exception as e:
return f"Error evaluating expression: {str(e)}"
# 3. Tool Declarations (OpenAI Compatible Schema)
TOOLS_SCHEMA = [
{
"type": "function",
"function": {
"name": "web_search",
"description": "Search the web for up-to-date information, current affairs, price facts, or real-time context.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query string."
}
},
"required": ["query"]
}
}
},
{
"type": "function",
"function": {
"name": "calculate",
"description": "Perform mathematical calculations for arithmetic queries.",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Math expression string, e.g. '10 * 21' or '3 * 90'."
}
},
"required": ["expression"]
}
}
}
]
# Mapping schema names to local Python functions
AVAILABLE_FUNCTIONS = {
"web_search": web_search,
"calculate": calculate
}
# 4. Agent Loop Implementation
def run_agent(user_prompt: str, max_iterations: int = 3):
messages = [
{
"role": "system",
"content": (
"You are an autonomous AI Agent equipped with tools. "
"Use 'web_search' for current facts or live information. "
"Use 'calculate' for mathematical or arithmetic expressions."
)
},
{"role": "user", "content": user_prompt}
]
for iteration in range(1, max_iterations + 1):
print(f"\n--- [Iteration {iteration}] Calling Model ---")
response = client.chat.completions.create(
model="grok-beta",
messages=messages,
tools=TOOLS_SCHEMA,
tool_choice="auto"
)
response_message = response.choices[0].message
messages.append(response_message)
# Check if the model requested tool execution
tool_calls = response_message.tool_calls
if not tool_calls:
# Model has finalized its response without needing further tools
print("Model produced final answer without tool calls.")
return response_message.content
# Process tool calls emitted by the model
for tool_call in tool_calls:
function_name = tool_call.function.name
function_args = ast.literal_eval(tool_call.function.arguments)
print(f"[Tool Selection]: {function_name} with arguments {function_args}")
target_function = AVAILABLE_FUNCTIONS.get(function_name)
if target_function:
# Execute the underlying Python tool
tool_output = target_function(**function_args)
# Append tool results back into message history
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": tool_output
})
else:
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": f"Error: Tool '{function_name}' not defined."
})
return "Agent reached maximum iteration limit before completing the request."
# 5. CLI Execution Loop
if __name__ == "__main__":
while True:
prompt = input("\nAsk something (or type 'exit'): ")
if prompt.lower() == "exit":
break
output = run_agent(prompt)
print(f"\nAgent Answer:\n{output}")
7. Execution Walkthrough & Debugging Insights
Testing the agent demonstrates how tool routing functions under different user intents:
Query 1: Information Retrieval ("Who is Pratyush Narayan?")
- Model Step: Inspects query; determines it requires dynamic context.
- Tool Selection: Invokes
web_search(query="Pratyush Narayan"). - Execution: Tavily fetches recent web snippets.
- Final Response: Returns a contextually verified summary of Pratyush Narayan's career and online presence.
CODE
Ask something: Who is Pratyush Narayan?
--- [Iteration 1] Calling Model ---
[Tool Selection]: web_search with arguments {'query': 'Pratyush Narayan'}
--- [Iteration 2] Calling Model ---
Model produced final answer without tool calls.
Agent Answer:
Pratyush Narayan is a software engineer and educator known for teaching AI Engineering...
Query 2: Exact Math Computation ("What is 3 * 90?")
- Model Step: Recognizes numerical computation requirement.
- Tool Selection: Invokes
calculate(expression="3 * 90"). - Execution: Python AST evaluator calculates
270. - Final Response: Synthesizes output accurately without numeric hallucination.
CODE
Ask something: What is 3 * 90?
--- [Iteration 1] Calling Model ---
[Tool Selection]: calculate with arguments {'expression': '3 * 90'}
--- [Iteration 2] Calling Model ---
Model produced final answer without tool calls.
Agent Answer:
3 * 90 = 270
8. Summary and Key Takeaways
- Core AI Agent Definition: An AI Agent consists of a Base LLM + Tools/APIs. It bridges conversational models with deterministic actions in external systems.
- Why Tools Matter: Raw LLMs struggle with precise mathematical calculations and static context limitations. Tools delegate math to deterministic code and search to external APIs.
- The Role of Tool Schemas: The
descriptionfield within the JSON Tool Schema acts as the critical semantic instruction telling the LLM when and why to trigger a specific tool. - Standard Tool Flow: Passing
tools=TOOLS_SCHEMAandtool_choice="auto"grants the model autonomy to decide whether to invoke a tool or reply directly. - Preventing Infinite Execution: Enforce iteration limits () within the agent execution loop to avoid unexpected API token burn on impossible prompts.
- Bridge to LangGraph: Hand-writing tool loops reveals the complexity of managing multi-agent systems manually. Frameworks like LangGraph exist to orchestrate, organize, and manage these stateful multi-agent workflows at production scale.