Handling enum and union types in LLM structured output

When you constrain model output with JSON Schema, enum and union types in structured output schemas are where most validation pipelines break. The model may emit a string not in the enum, or a union shape that doesn’t match any branch, leaving your parser to choke on perfectly plausible text. This guide gives you ordered steps to define, request, and validate those types end to end with Pydantic and Zod, so your pipeline fails loud instead of silent.

Step 1: Define enum and union types in your schema

Start with the data model. Enums map to a fixed set of string literals; unions need a discriminator to be reliably emitted by LLMs. Plain structural unions (anyOf) without a tag force the model to infer which branch it produced, which it does poorly.

Enums with Pydantic

Use Literal for strict enum values. Pydantic v2 emits a JSON Schema enum cleanly and validates against it.

from typing import Literal
from pydantic import BaseModel

class Ticket(BaseModel):
    priority: Literal["low", "medium", "high"]
    status: Literal["open", "closed", "pending"]

If you prefer Enum, subclass str so the serialized value is the string, not the Python enum name:

from enum import Enum
class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"

Enums with Zod

import { z } from "zod";

const Ticket = z.object({
  priority: z.enum(["low", "medium", "high"]),
  status: z.enum(["open", "closed", "pending"]),
});

Unions: always use discriminated unions

Give the model an explicit type field to switch on. Pydantic uses Field(discriminator=...); Zod uses z.discriminatedUnion.

from typing import Union, Literal
from pydantic import BaseModel, Field

class Cat(BaseModel):
    type: Literal["cat"]
    lives_left: int

class Dog(BaseModel):
    type: Literal["dog"]
    bark_volume: int

class Pet(BaseModel):
    animal: Union[Cat, Dog] = Field(discriminator="type")

const Cat = z.object({ type: z.literal("cat"), lives_left: z.number() });
const Dog = z.object({ type: z.literal("dog"), bark_volume: z.number() });
const Pet = z.discriminatedUnion("type", [Cat, Dog]);

The discriminator turns ambiguous shapes into enum and union types in structured output schemas that the model can actually target.

Step 2: Generate provider-compatible JSON Schema

Call model_json_schema() (Pydantic) or use zod-to-json-schema for Zod. Strip fields some providers reject, like $schema or title, if your endpoint is strict.

schema = Pet.model_json_schema()
schema.pop("$schema", None)
schema.pop("title", None)

For the discriminated union, the schema contains oneOf with required: ["type"] and properties.type.enum. A snippet looks like:

{
  "type": "object",
  "properties": {
    "animal": {
      "oneOf": [
        {
          "type": "object",
          "properties": {
            "type": { "enum": ["cat"], "type": "string" },
            "lives_left": { "type": "integer" }
          },
          "required": ["type", "lives_left"]
        },
        {
          "type": "object",
          "properties": {
            "type": { "enum": ["dog"], "type": "string" },
            "bark_volume": { "type": "integer" }
          },
          "required": ["type", "bark_volume"]
        }
      ]
    }
  }
}

That explicit shape is far more reliable than anyOf with structural matching.

Step 3: Request structured output from the model

Use response_format with json_schema type. Set strict: true if the provider supports it; this forces compliance with the schema.

from openai import OpenAI

client = OpenAI()  # or point to compatible gateway
completion = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Describe a pet: a dog that barks at 80dB."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "pet",
            "strict": True,
            "schema": schema,
        },
    },
)
raw = completion.choices[0].message.content

If your model does not support strict, omit it and rely on validation in Step 5. The same pattern works in TypeScript with the OpenAI SDK.

Step 4: Route through an OpenAI-compatible gateway

If you swap models frequently, point the client at a gateway that speaks the OpenAI protocol. n4n.ai exposes one OpenAI-compatible endpoint covering 240+ models, automatically falls back when a provider is rate-limited or degraded, honors client routing directives, forwards provider cache-control hints, and reports per-token usage metering, so the schema you built in Step 2 stays constant while the backend changes.

client = OpenAI(
    base_url="https://api.n4n.ai/v1",
    api_key="your-key",
)

You can pin a model or let the gateway choose without rewriting schema code.

Step 5: Validate and coerce the response

Never trust the raw string. Parse it back into your model. Pydantic raises ValidationError on enum mismatch or missing discriminator.

import json
from pydantic import ValidationError

try:
    pet = Pet.model_validate(json.loads(raw))
except ValidationError as e:
    # Log and either retry with stronger prompt or fallback to manual parse
    print("Schema violation:", e)
    raise

Zod:

const result = Pet.safeParse(JSON.parse(raw));
if (!result.success) {
  console.error(result.error);
  throw new Error("Invalid structured output");
}

If the model returns "priority": "urgent" against your enum, validation fails fast. Add a retry with a reminder of allowed values, or map unknown to a default in post-processing if your domain allows. For integer fields, some models emit 3.0; coerce with Pydantic coerce_numbers_to_int or Zod .transform(Math.trunc).

Step 6: Verify success with a concrete test

Write a test that asserts both branches of the union and every enum value parse correctly. Use known-good JSON fixtures.

def test_pet_union():
    dog_json = '{"animal": {"type": "dog", "bark_volume": 80}}'
    cat_json = '{"animal": {"type": "cat", "lives_left": 9}}'
    assert Pet.model_validate_json(dog_json).animal.type == "dog"
    assert Pet.model_validate_json(cat_json).animal.type == "cat"

def test_enum_reject():
    import pytest
    with pytest.raises(ValidationError):
        Pet.model_validate_json('{"animal": {"type": "fish", "x": 1}}')

def test_ticket_enum():
    t = Ticket(priority="low", status="open")
    assert t.priority == "low"
    with pytest.raises(ValidationError):
        Ticket(priority="critical", status="open")

Run pytest. Green means your enum and union types in structured output schemas survive round-trips. In TypeScript, use vitest with equivalent expect(() => Pet.parse(bad)).toThrow().

Practical edge cases

• Enum-adjacent strings: Prompt with the exact enum list; set strict to force compliance. If a provider lacks strict mode, validate and map.
• Nested unions: The discriminator must be present at each Union level. Don’t nest anonymous unions.
• Integer vs number: JSON Schema type: integer trips models that emit 3.0. Coerce explicitly.
• Cache hints: When using a gateway that forwards cache-control, mark stable schema portions with {"cache_control": {"type": "ephemeral"}} in the request to cut token cost on repeated calls.

Closing checklist

You defined enums as literals, used discriminated unions for branch selection, generated minimal JSON Schema, requested it strictly, validated the response, and tested both branches. That covers the full lifecycle of enum and union types in structured output schemas. Do this and your LLM output stops being a guess.