GenAI with Python & LangChain: Chapter 9 — Structured Output in LangChain

By default, Large Language Models (LLMs) return raw string outputs wrapped in message abstractions. While plain text responses are sufficient for chatbots, production software applications demand predictable, strictly typed data structures—such as JSON objects, Python dictionaries, or typed domain model instances—to enable reliable machine-to-machine integration, API payloads, and agentic workflows. This chapter explores how to enforce deterministic schema formatting using LangChain's native with_structured_output interface combined with Pydantic models.

1. The Core Problem: Unstructured String Outputs

When querying an LLM using standard invocation patterns, the response payload is returned as an AIMessage where the content attribute is a continuous string.

Why String Outputs Fail in Software Pipelines

If an application needs to extract distinct data points (e.g., a user's name, email address, and age) from unstructured text, relying on default LLM output introduces significant fragility:

1. No Type Guarantees: Numerical values like age are returned as text strings ("29") rather than native integers (29).
2. Parsing Fragility: Extracting values using string manipulation (split()) or regular expressions fails whenever the LLM slightly alters its formatting or conversational prefix (e.g., returning "Here is the requested info: Name:..." instead of raw key-value pairs).
3. Lack of Attribute Access: You cannot programmatically access properties via object attributes (e.g., response.name).

Baseline Demonstration: Naive LLM Extraction

import os
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI

# Load environment variables (OPENAI_API_KEY)
load_dotenv()

# Initialize standard Chat Model
llm = ChatOpenAI(model="gpt-4o", temperature=0)

# Unstructured multi-line input text
input_text = """
Hello, my name is Prateek Mishra. 
My email is prateek29mishra@gmail.com, and my age is 29.
"""

# Prompting the LLM directly
query = f"Please give me only name, email, and age from this text:\n{input_text}"
response = llm.invoke(query)

print("Raw Content:", response.content)
print("Content Type:", type(response.content))

Output:

Raw Content: Name: Prateek Mishra
Email: prateek29mishra@gmail.com
Age: 29
Content Type: <class 'str'>

Note: Attempting to query response.content.email throws an AttributeError because response.content is simply Python's built-in str type.

2. Schema Foundation: Data Modeling with Pydantic

To force an LLM to generate outputs adhering to a rigid schema, we define our desired data structure using Pydantic (pydantic.BaseModel).

Pydantic is Python's standard library for data validation and settings management using Python type annotations. LangChain leverages Pydantic models to construct underlying JSON Schemas, which are then passed to the LLM via native function calling or tool-calling APIs.

The Critical Role of Field(description=...)

Simply defining field names and types is often insufficient for an LLM to understand what data to place into a given field. By using Pydantic’s Field function with explicit description strings, you supply semantic contextual cues directly to the model.

from pydantic import BaseModel, Field

class ResponseStructure(BaseModel):
    name: str = Field(description="The full complete name of the user")
    email: str = Field(description="The email address of the user")
    age: int = Field(description="The age of the user as an integer")

Conceptually, the Field(description=...) metadata is transformed into JSON Schema properties:

{
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "The full complete name of the user"
    },
    "email": {
      "type": "string",
      "description": "The email address of the user"
    },
    "age": {
      "type": "integer",
      "description": "The age of the user as an integer"
    }
  },
  "required": ["name", "email", "age"]
}

3. Implementing Structured Output via with_structured_output

LangChain provides the high-level .with_structured_output() method on chat models. This method wraps the LLM, automatically binding the provided schema and parsing the raw LLM output directly into a validated instance of your Pydantic model.

Step-by-Step Implementation

from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI

# 1. Define the Pydantic Schema
class ResponseStructure(BaseModel):
    name: str = Field(description="Complete name of the user")
    email: str = Field(description="Email address of the user")
    age: int = Field(description="Age of the user")

# 2. Instantiate the Base LLM
llm = ChatOpenAI(model="gpt-4o", temperature=0)

# 3. Bind the Schema to create a Structured LLM
structured_llm = llm.with_structured_output(ResponseStructure)

# 4. Invoke the Structured LLM
input_text = """
Hello, my name is Prateek Mishra. 
My email is prateek29mishra@gmail.com, and my age is 29.
"""

response = structured_llm.invoke(f"Extract details from this text: {input_text}")

# 5. Verify and Access Attributes
print(f"Object Type: {type(response)}")
print(f"Name: {response.name}")
print(f"Email: {response.email}")
print(f"Age: {response.age} (Type: {type(response.age)})")

Output:

Object Type: <class '__main__.ResponseStructure'>
Name: Prateek Mishra
Email: prateek29mishra@gmail.com
Age: 29 (Type: <class 'int'>)

Exporting Pydantic Objects to Python Dictionaries

If your backend requires plain JSON or standard Python dictionaries (e.g., for REST API responses or database writes), call .model_dump() on the returned Pydantic instance:

# Convert Pydantic model instance to a standard Python dictionary
dict_response = response.model_dump()

print(dict_response)
print(f"Dictionary Type: {type(dict_response)}")

Output:

{'name': 'Prateek Mishra', 'email': 'prateek29mishra@gmail.com', 'age': 29}
Dictionary Type: <class 'dict'>

4. Advanced Schemas: Nested Objects and Multi-Item Extraction

Real-world applications frequently require extracting lists of items or complex nested hierarchies (e.g., querying for multiple items, historical records, or relational data).

Evolution from Single Items to Collections

Suppose we ask the model for movie recommendations. Modeling a single movie object requires title and year. However, if we request multiple movies, passing a single-item class to .with_structured_output() forces the model to return only one instance.

To handle collections cleanly, we combine standard Python typing (typing.List) with a container Pydantic class.

Code Implementation: Nested Schema for Multiple Movies

from typing import List
from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI

# Single Entity Schema
class Movie(BaseModel):
    title: str = Field(description="The title of the movie")
    year: int = Field(description="The release year of the movie")

# Container Schema for Collections
class AllMovies(BaseModel):
    movies: List[Movie] = Field(description="A list of multiple movie recommendations")

# Initialize Model and Bind Container Schema
llm = ChatOpenAI(model="gpt-4o", temperature=0)
movie_llm = llm.with_structured_output(AllMovies)

# Invoke query requesting multiple items
query = "Give me 3 top sci-fi movies."
response = movie_llm.invoke(query)

# Inspect Structured Nested Output
print(f"Container Type: {type(response)}")
print(f"Number of Movies Returned: {len(response.movies)}\n")

for idx, movie in enumerate(response.movies, start=1):
    print(f"{idx}. {movie.title} ({movie.year}) -> Type: {type(movie)}")

Output:

5. Architectural Flow Walkthrough

The following diagram illustrates the complete execution pipeline from raw input to strongly typed domain objects inside your application:

6. Real-World Practical Pitfalls & Edge Cases

When working with structured output in production, software engineers often encounter specific traps:

Pitfall 1: Binding the Item Schema Instead of the Collection Schema

• Problem: Requesting a list of items while passing a single item schema (e.g., llm.with_structured_output(Movie)) results in the LLM returning only the first item or throwing a schema validation error.
• Solution: Always create a wrapper model containing a List[Item] field (e.g., AllMovies) when querying for variable numbers of entities.

Pitfall 2: Missing Field Descriptions

• Problem: Defining fields with ambiguous names like data: str without a Field(description=...) causes the model to guess what to extract, leading to inaccurate outputs.
• Solution: Annotate every field with clean, explicit context inside Field(description="...").

Pitfall 3: Unsupported Model Provider APIs

• Problem: .with_structured_output() relies on native tool calling / function calling capabilities under the hood. Older models or open-source LLMs without tool-calling capabilities will fail.
• Solution: Ensure your target model natively supports tool calling (e.g., OpenAI GPT-4o/GPT-3.5-Turbo, Anthropic Claude 3/3.5, or mistral-large).

7. Comparative Analysis of Extraction Strategies

8. Summary and Key Takeaways

• Default LLM Output: Standard LLM calls return unstructured strings (AIMessage.content), which are error-prone and unsuitable for deterministic software backends.
• Pydantic Schemas: Pydantic's BaseModel defines structural contracts, field types, and semantic context required by LLMs to parse information correctly.
• Field Context: Providing precise semantic descriptions via Field(description="...") guides the LLM on what specific data each field requires.
• Native Binding: The llm.with_structured_output(Schema) method binds Pydantic models directly to modern LLM function-calling capabilities.
• Handling Arrays/Lists: Multi-item extraction requires wrapping primitive schemas in a dedicated container model using typing.List (e.g., movies: List[Movie]).
• Data Export: Returned Pydantic instances support direct dot-notation attribute access (obj.field) and can be serialized to native Python dictionaries via .model_dump().
• Agentic Prerequisite: Enforcing structured output is foundational for multi-agent systems and Machine-to-Machine (M2M) communication pipelines, where the output of one LLM node serves as structured input to another.