Ep:05-Pydantic+JSON | Structured Outputs for AI Engineering

This lecture introduces the shift from human-readable conversational text to machine-readable structured outputs. While raw text responses from LLMs are ideal for interactive chats, modern production AI agents must interface cleanly with downstream code pipelines (written in Python, Java, C++, Go, etc.). By pairing Pydantic schema definitions with JSON output formatting, developers can transform unpredictable LLM outputs into strictly typed, production-ready data streams.

1. The Unstructured Text Problem in Production

In basic chat applications, an LLM takes a string prompt and returns a raw text string response. For human consumption, text output works well—humans easily parse paragraphs, ignore conversational filler, and extract contextual meaning.

However, in enterprise environments, 90% of AI agents do not output data to human readers. Instead, their outputs are consumed directly by downstream software modules running at scale (e.g., executing 1,000,000 times per day).

The String Parsing Dilemma

When an LLM returns a response like:

"Sure! Let me help you with that. The customer's name is Pratyush, their email is abc@gmail.com, and their issue belongs to Electronics."

The downstream system must execute complex string matching, regex searching, or substring slicing to isolate key variables. In computer science, string parsing is notoriously brittle. If the LLM randomly alters its conversational preamble or formatting (e.g., using bullet points instead of bold text), the downstream parser breaks, leading to runtime crashes.

Note on Prerequisites: Jumping into structured outputs without understanding core API calls is like arriving late to a dinner party during the main course and asking the host for tea. Ensure you have completed the earlier episodes on basic LLM initialization and environment setup before proceeding.

2. Real-World Analogy: The Watchman’s Logbook

To understand the difference between unstructured text and structured data, consider a security guard at a building entrance tasked with logging visitors.

Real-Life Analogy

Unstructured Logging (Conversational String):
The guard writes a narrative paragraph in his diary:
"Pratyush arrived at around 2:00 PM today driving a vehicle with plate number DL-XX-7338."
If the building owner wants to scan 500 entries to find a specific license plate, they must read through hundreds of words of filler text.

Structured Logging (JSON Schema):
The guard uses a structured table/logbook with fixed columns:

Visitor Name	Arrival Time	Vehicle Plate Number
Pratyush	02:00 PM	DL-XX-7338

Now, querying the logbook takes milliseconds because the key-value structure is defined and consistent.

3. Naive Prompting vs. Downstream Code Requirements

Consider a real-world Customer Support Agent processing incoming emails into support tickets.

Scenario: Raw Email Input

Hi, my name is Pratyush. I bought an iPhone from your store and it stopped working. 
My address is Delhi and I broke up with my girlfriend yesterday. 
Please contact me on abc@gmail.com or call 82155XXXXX. Regards, Pratyush.

Naive Prompt Approach

If you pass this raw text to an LLM asking it to extract the user's details, a typical unstructured response looks like this:

Sure! Here is the extracted information:
- Name: Pratyush
- Email: abc@gmail.com
- Issue: Electronics / Hardware Fault
- Phone: 82155XXXXX

Why Downstream Code Struggles

If a downstream service written in C++ or Java needs to store this data in a database, it must parse out - Name: or **Name:**. If the LLM temperature shifts or the model version changes, the output formatting shifts unpredictably:

Writing regex algorithms to reliably account for all variations across millions of calls is virtually impossible.

4. The Core Solution: Pydantic + JSON

To guarantee consistent formatting, we enforce two standard tooling layers:

1. JSON (JavaScript Object Notation): A universal, lightweight, language-agnostic data-interchange format structured as explicit key-value pairs.
2. Pydantic: A data validation and settings management library for Python. It enforces type hints at runtime and generates standard JSON schemas automatically.

With JSON outputs, downstream code bypasses complex regex logic entirely. A developer can extract variables instantly using simple key lookups or programmatic dot-notation (e.g., ticket.name, ticket.email).

5. Step-by-Step Evolution: Building a Structured Ticket Parser

Let's build a structured support-ticket extraction agent step-by-step using Python, Groq, and Pydantic.

Step 1: Environment Setup

Install the necessary dependencies inside your active virtual environment (.venv):

uv add groq python-dotenv pydantic

Step 2: Define the Pydantic Data Model

We declare a class inheriting from Pydantic’s BaseModel. This defines the exact schema and data types we expect from the LLM.

from pydantic import BaseModel

class SupportTicket(BaseModel):
    name: str
    email: str
    issue: str

Step 3: Extract the Schema & Enforce JSON Format

Convert the Pydantic model into a standard JSON schema that can be injected into the system prompt and client payload parameters:

# Generate JSON Schema from the Pydantic Model
ticket_schema = SupportTicket.model_json_schema()

# Define the strict response format expected from the API call
response_format = {
    "type": "json_object"
}

Step 4: Construct System & User Prompts

The LLM must be explicitly instructed in the system prompt to output JSON matching the provided schema.

system_prompt = f"""
You are an expert data extraction agent. 
Extract the relevant support ticket details from the incoming text.
Return your output strictly as a JSON object matching this schema:
{ticket_schema}
"""

user_text = """
Hello, my name is Pratyush. I purchased an iPhone from your store and it stopped working.
My address is Delhi and my email is abc@gmail.com.
"""

messages = [
    {
        "role": "system",
        "content": system_prompt
    },
    {
        "role": "user",
        "content": f"Extract details from this customer message: {user_text}"
    }
]

Step 5: Execute the LLM Call & Parse JSON

Pass the messages and response_format to the LLM client, then parse the raw JSON string back into a strongly-typed Pydantic object using json.loads().

import os
import json
from groq import Groq
from dotenv import load_dotenv

load_dotenv()
client = Groq(api_key=os.getenv("GROQ_API_KEY"))

# API Request
response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=messages,
    response_format=response_format,
    temperature=0.1
)

# Extract raw JSON string output
raw_json_output = response.choices[0].message.content
print("Raw JSON Output from LLM:")
print(raw_json_output)

# Step 6: Load JSON into Python Dictionary and instantiate Pydantic object
json_data = json.loads(raw_json_output)
structured_ticket = SupportTicket(**json_data)

# Accessing fields via programmatic dot-notation
print("\nParsed Data Fields:")
print(f"Name: {structured_ticket.name}")
print(f"Email: {structured_ticket.email}")
print(f"Issue: {structured_ticket.issue}")

6. End-to-End System Flow Architecture

The data pipeline transitions from unstructured inputs to structured objects cleanly:

7. Real-World Debugging & Pitfalls

When implementing structured outputs in production, developers frequently encounter three specific errors:

Pitfall 1: Forgetting BaseModel Inheritance

• Symptom: Python throws AttributeError: type object 'Ticket' has no attribute 'model_json_schema'.
• Cause: The class was declared as class Ticket: instead of inheriting from Pydantic's base class: class Ticket(BaseModel):.
• Fix:
  PYTHON

  Copy

  # WRONG
  class Ticket:
      name: str
  
  # CORRECT
  class Ticket(BaseModel):
      name: str
  

Pitfall 2: Missing the Keyword "JSON" in the System Prompt

• Symptom: API error response: Messages must contain the word 'json' in some form to use response_format.
• Cause: When explicitly passing response_format={"type": "json_object"}, API providers (OpenAI, Groq) enforce an automated guardrail requiring the string "json" to appear inside your prompt array.
• Fix: Ensure your system prompt explicitly includes the phrase "in JSON format".

Pitfall 3: Dictionary Key Mismatches During Unpacking

• Symptom: Pydantic ValidationError during object initialization (SupportTicket(**json_data)).
• Cause: The LLM hallucinated a key name (e.g., returning customer_name instead of name).
• Fix: Use strict schema definitions, low temperature settings (0.0 to 0.2), and clear field naming inside your system prompt.

8. Summary Comparison: Output Paradigms

9. Mini-Project Assignment: Automated HR Resume Parser

To consolidate your learning, build a production-grade HR Resume Parser & Fit Evaluator.

Problem Context

HR departments spend hundreds of hours downloading resumes (PDF/Word format), scanning text, matching candidate profiles against Job Descriptions (JDs), and estimating candidate suitabilities. You need to build an automated software agent that handles this pipeline end-to-end.

Requirements & Functional Specification

1. Input Data:

   • Raw resume text (containing experience, technical skills, and candidate contact details).
   • Target Job Description criteria provided by HR (e.g., Required Experience: 2 years; Required Skills: Python, SQL, LLMs; Target Projects: ML/AI pipelines).

2. Schema Definition (Pydantic): Define a model to capture candidate data and match results:

   PYTHON

   Copy

   from pydantic import BaseModel
   from typing import List
   
   class ResumeEvaluation(BaseModel):
       candidate_name: str
       extracted_skills: List[str]
       years_of_experience: float
       matching_score_percentage: float
       good_fit: bool
       summary_reasoning: str
   

3. Core Logic:

   • Extract fields strictly using the defined schema.
   • Compare candidate experience against HR job criteria.
   • Compute a matching_score_percentage (e.g., 82.0% for strong fits, 38.0% for weak fits).

4. Downstream Decision Action:

   • If matching_score_percentage >= 70.0, trigger an automated output: "Candidate recommended for HR screening call."
   • Otherwise: "Candidate fit percentage too low. Hold application."

10. Summary and Key Takeaways

• Unstructured text is built for humans, not machines: Standard LLM text responses contain conversational filler that breaks downstream code parsers.
• Production AI agents demand structured schemas: Over 90% of enterprise AI outputs are consumed by secondary programs (C++, Java, Python services) where string parsing is inefficient and risky.
• JSON is the industry standard for inter-system communication: Wrapping LLM responses in JSON ensures language-agnostic data sharing across servers and services worldwide.
• Pydantic automates data validation: By inheriting from BaseModel, Pydantic generates standard JSON schemas directly from Python type declarations.
• Low temperatures stabilize schemas: Keep LLM temperature settings low (0.0–0.2) when extracting JSON to prevent schema drift and key hallucinations.
• Prompt guardrails are strictly enforced: When requesting response_format={"type": "json_object"}, modern API endpoints require the keyword "json" to appear explicitly in the prompt text.
• Dot-notation simplifies downstream parsing: Using Pydantic objects allows developers to access output fields cleanly (ticket.name, ticket.email) without running regex scripts.