Picture a triage bot. It reads a customer's bug report, asks an LLM to turn it into a structured ticket, and files it in the tracker. It handles 40 reports without a problem. On the 41st, it crashes because the ticket has no severity, and the crash happens several steps away from where the LLM's answer arrived. Nothing along the way noticed the ticket was incomplete.

The root problem is that the LLM's answer comes from outside your program, so you can't count on its shape. Every piece of code that touches it has to either re-check it or trust it blindly.

Pydantic is a Python library that fixes this with a single checkpoint. You describe the shape you expect, once. Incoming data is checked against it. What comes out is either a clean object you can trust, or a clear error right at the door.

Checking by hand doesn't remove the failure, it just moves it

Here's the shape of the manual check most people write first, working directly on the dict the LLM handed back:

def route_ticket(llm_output: dict) -> None:
    severity = llm_output.get("severity")
    if severity == "critical":
        page_oncall(llm_output["component"])
    # ... eight more lines down, something else reads llm_output["steps_to_reproduce"]

This works until the LLM spells severity "urgent" instead of one of the four values your paging logic understands, or leaves out steps_to_reproduce entirely. Then the failure happens wherever that field is first touched: this function, a report formatter three files away, or a scheduled job at 2am that pages nobody about the actual bug.

The dict never had a shape to begin with. It's just keys and values.

A dataclass declares a shape. It doesn't enforce one.

If you haven't used one before: a dataclass is a decorator that takes a class with type-annotated fields and generates __init__, __repr__, and equality for you, so you're not hand-writing that boilerplate. It looks like the fix:

from dataclasses import dataclass

@dataclass
class Ticket:
    title: str
    severity: str
    component: str
    steps_to_reproduce: list[str]

Now try constructing one from the same bad LLM output:

>>> Ticket(title="Login broken", severity=None, component="auth", steps_to_reproduce="none given")
Ticket(title='Login broken', severity=None, component='auth', steps_to_reproduce='none given')

It runs. severity: str said "this is a string," and Python let a None walk right past it. steps_to_reproduce: list[str] said "a list of strings," and it accepted a bare string instead. Type hints on a dataclass are read by your editor and by mypy, at edit time, on code you wrote. Nothing in a plain dataclass reads them at runtime, on data that showed up from outside your program, which is when they'd matter.

Pydantic: same class, but it checks at construction

A Pydantic BaseModel looks almost identical to write, and does the opposite at construction time: it checks.

from typing import Literal
from pydantic import BaseModel, Field

class Ticket(BaseModel):
    title: str = Field(min_length=5, max_length=80)
    severity: Literal["low", "medium", "high", "critical"]
    component: str
    steps_to_reproduce: list[str] = Field(min_length=1)
    reporter_email: str

Literal[...] restricts severity to exactly those four strings; anything else is rejected when the model is constructed. Field(min_length=1) on the list means an empty steps_to_reproduce is rejected, not silently accepted as "no steps." Now feed it the same bad data:

>>> Ticket(title="Login broken", severity=None, component="auth", steps_to_reproduce="none given")
Traceback (most recent call last):
  ...
pydantic_core._pydantic_core.ValidationError: 2 validation errors for Ticket
severity
  Input should be 'low', 'medium', 'high' or 'critical' [type=literal_error, input_value=None, input_type=NoneType]
steps_to_reproduce
  Input should be a valid list [type=list_type, input_value='none given', input_type=str]

Both problems, named, at the moment the object was built. No downstream function ever sees this Ticket, because it never gets created. The dict version would have let the same bad data flow silently into route_ticket(), format_report(), and anything else that touched it.

Parse, don't validate

This idea has a name: parse, don't validate. Validating means checking data in place, over and over, at every site that uses it. Parsing means converting untrusted input into a new, more specific type once, at the point it enters your program, so everything after that can simply trust the type it's holding.

A dict passed around your codebase never stops being "a dict that might have a severity key." A Ticket instance, once it exists, is guaranteed to have a severity that's one of four strings and a non-empty steps_to_reproduce list, because there was no way to construct one otherwise. The check happens once, at the door. That door is the trust boundary: the point where data from outside your program becomes data your code is allowed to rely on. Everything on the other side gets to stop worrying about it.

Approach

Checks types at runtime?

Where a bad value surfaces

What you write

Dict + .get() / if checks

No

Wherever the field is first used, could be far downstream

A defensive check in every function that touches the data

@dataclass

No (editor/mypy only, not at runtime)

Same as above

Field names and types declared, but not enforced

Pydantic BaseModel

Yes

At construction, right where the untrusted data enters

One class declaration, checked automatically every time

Structured outputs narrow the shape, not your rules

OpenAI's structured outputs feature lets you hand it a Pydantic model directly and get back an instance of that model instead of raw text you have to parse yourself:

from openai import OpenAI, LengthFinishReasonError

client = OpenAI()

def extract_ticket(report_text: str) -> Ticket | None:
    try:
        completion = client.beta.chat.completions.parse(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": "Extract a bug ticket from the report below."},
                {"role": "user", "content": report_text},
            ],
            response_format=Ticket,
            max_tokens=200,
        )
    except LengthFinishReasonError:
        print("Response was cut off before the JSON closed. Raise max_tokens and retry.")
        return None

    message = completion.choices[0].message
    if message.refusal:
        print(f"Model declined to answer: {message.refusal}")
        return None

    return message.parsed

Feed it a real report:

report_text = """
Login button does nothing when I click it in Safari 17. Tried three
times, no error message, page just sits there. Blocks me from using
the app at all. - Dana ([email protected])
"""

ticket = extract_ticket(report_text)

Conforming case, what message.parsed comes back as:

Ticket(
    title='Login button unresponsive in Safari 17',
    severity='high',
    component='auth',
    steps_to_reproduce=[
        'Open the app in Safari 17',
        'Click the login button',
        'Observe that nothing happens, no error shown',
    ],
    reporter_email='[email protected]',
)

response_format=Ticket converts your model into a schema and constrains generation to match it: type and the four allowed severity values are enforced, so you won't get a fifth severity string back. That's real progress over prompting a model with "please return JSON" and hoping.

It's not a full replacement for validation, though. Two gaps remain.

Truncation still produces garbage

Set max_tokens=40 on that same call, too low for the schema, and the model gets cut off mid-way through writing something like this (illustrative; the exact cut point depends on tokenization):

{"title": "Login button unresponsive in Safari 17", "severity": "high", "component": "auth", "steps_to_repro

Not valid JSON, and it can't become a Ticket. client.beta.chat.completions.parse() detects this itself and raises LengthFinishReasonError, which is why extract_ticket() catches it explicitly instead of letting a cryptic JSONDecodeError bubble up from somewhere less obvious.

OpenAI's schema enforcement doesn't cover every rule on your model

Our Ticket has length rules: a title of at least 5 characters, and at least one reproduction step. OpenAI's structured outputs enforce type and enum values during generation but may not apply those rules, so the model can still emit a 3-character title like "Bug" or an empty steps_to_reproduce list. Pydantic then rejects it with a ValidationError when the SDK builds the Ticket.

Retry a bounded number of times

The ValidationError is raised inside the client.beta.chat.completions.parse() call itself, because the SDK builds the Ticket before handing anything back. extract_ticket() above only catches truncation, so this error would propagate straight out of it. The model's output varies from call to call, so the same request often passes on a second try. The usual pattern is a small, bounded retry loop:

  • Catch ValidationError narrowly, not a bare Exception.

  • Feed the error text back to the model. It names the field and the bad value, which is what the model needs to correct itself.

  • Cap the attempts at two or three, then give up loudly (return None, or route the report to a human) instead of looping forever.

  • Don't retry a refusal. Asking again doesn't change the model's answer.

Only the loop and the new except branch differ from extract_ticket(); the truncation and refusal handling stay as before:

from pydantic import ValidationError

# messages = [system, user] as in extract_ticket()
for _ in range(max_attempts):  # max_attempts = 3
    try:
        completion = client.beta.chat.completions.parse(
            model="gpt-4o", messages=messages, response_format=Ticket, max_tokens=200,
        )
    except ValidationError as err:
        messages.append({
            "role": "user",
            "content": f"That ticket failed validation:\n{err}\nReturn a corrected ticket.",
        })
        continue
    # ... LengthFinishReasonError and refusal handling, then return message.parsed

print(f"No valid ticket after {max_attempts} attempts.")
return None

Takeaway

  • Pydantic turns data you don't control (an LLM response, a request body, a config file) into an object you can trust, or refuses to build one.

  • A dataclass declares a shape but never checks it at runtime; a Pydantic BaseModel checks every field the moment you construct one. That's "parse, don't validate" in practice: once a Ticket exists, everything downstream can trust its shape without re-checking.

  • OpenAI's structured outputs enforce types and enum values during generation, but not length rules, and a truncated response still fails. Handle a refusal, handle truncation, and let Pydantic's field constraints catch the rest, with a bounded retry that feeds the ValidationError back to the model.