Zod schemas for validating LLM API responses

LLM APIs lie. Not maliciously, but the JSON you get back from a model endpoint rarely matches the TypeScript type you assumed. Using Zod schemas for zod llm api response validation turns that ambiguity into a hard runtime check, so a malformed completion fails fast instead of poisoning your app logic.

Step 1: Install dependencies and configure the client

Pull the OpenAI SDK and Zod. The OpenAI package works against any OpenAI-compatible endpoint, which matters when you aggregate models behind one gateway.

npm install openai zod

Initialize the client. Point baseURL at your provider. If you route through n4n.ai, one OpenAI-compatible endpoint covers 240+ models, and the response shape matches the OpenAI schema your Zod parser expects.

import OpenAI from 'openai';
import { z } from 'zod';

const client = new OpenAI({
  baseURL: process.env.LLM_BASE_URL ?? 'https://api.openai.com/v1',
  apiKey: process.env.LLM_API_KEY!,
});

Keep the client singleton. Recreating it per request leaks connections and defeats keep-alive.

Step 2: Define the expected output shape with Zod

Write the schema before you write the prompt. The schema is the contract; the prompt is just a hint to the model.

Suppose you extract structured data from support tickets:

export const TicketSchema = z.object({
  priority: z.enum(['low', 'medium', 'high', 'critical']),
  category: z.enum(['billing', 'technical', 'account', 'other']),
  summary: z.string().min(10).max(200),
  tags: z.array(z.string()).max(5),
});

export type Ticket = z.infer<typeof TicketSchema>;

This is your first line of zod llm api response validation. The z.infer type gives you static safety; the runtime parse gives you dynamic safety.

Step 3: Request a constrained completion and parse it

Force JSON output with response_format: { type: 'json_object' }. Then parse the string content and validate.

async function extractTicket(emailText: string): Promise<Ticket> {
  const completion = await client.chat.completions.create({
    model: 'gpt-4o-mini',
    response_format: { type: 'json_object' },
    messages: [
      {
        role: 'system',
        content: 'Extract a support ticket. Respond only with JSON matching the schema.',
      },
      { role: 'user', content: emailText },
    ],
  });

  const raw = completion.choices[0]?.message?.content;
  if (!raw) throw new Error('Empty completion');

  const json = JSON.parse(raw);
  return TicketSchema.parse(json); // throws on mismatch
}

TicketSchema.parse throws a ZodError if the model drifts. That exception should bubble to a layer that can retry or fall back to a stronger model.

Step 4: Handle validation failures without crashing the pipeline

In production, use safeParse and decide on a fallback path. Never silently cast with as Ticket.

async function safeExtract(emailText: string): Promise<Ticket | null> {
  const completion = await client.chat.completions.create({
    model: 'gpt-4o-mini',
    response_format: { type: 'json_object' },
    messages: [{ role: 'user', content: emailText }],
  });

  const raw = completion.choices[0]?.message?.content ?? '{}';
  const result = TicketSchema.safeParse(tryParse(raw));
  if (!result.success) {
    console.error('Validation failed', result.error.issues);
    return null;
  }
  return result.data;
}

function tryParse(s: string) {
  try { return JSON.parse(s); } catch { return {}; }
}

Returning null lets the caller trigger a retry with a different model or a stricter prompt. That’s the core of resilient zod llm api response validation.

Step 5: Validate the full response envelope

Models and gateways occasionally return truncated payloads or unexpected usage fields. Validate the envelope too, especially when metering tokens.

const UsageSchema = z.object({
  prompt_tokens: z.number(),
  completion_tokens: z.number(),
  total_tokens: z.number(),
});

const ChatCompletionSchema = z.object({
  id: z.string(),
  choices: z.array(
    z.object({
      index: z.number(),
      message: z.object({
        role: z.string(),
        content: z.string().nullable(),
      }),
      finish_reason: z.string().nullable(),
    })
  ),
  usage: UsageSchema.optional(),
});

export type ChatCompletion = z.infer<typeof ChatCompletionSchema>;

Run the raw SDK response through ChatCompletionSchema.parse before you even look at choices[0].message.content. This catches provider-specific shape shifts early.

Step 6: Compose schemas for polymorphic LLM outputs

Real workflows return variants: either a tool call or a natural language reply. Use z.discriminatedUnion to model that.

const ToolCallSchema = z.object({
  type: z.literal('tool'),
  tool: z.string(),
  args: z.record(z.unknown()),
});

const TextReplySchema = z.object({
  type: z.literal('text'),
  content: z.string(),
});

const AgentOutputSchema = z.discriminatedUnion('type', [
  ToolCallSchema,
  TextReplySchema,
]);

function handleAgent(raw: unknown) {
  const out = AgentOutputSchema.parse(raw);
  if (out.type === 'tool') {
    return callTool(out.tool, out.args);
  }
  return replyUser(out.content);
}

Discriminated unions keep your switch exhaustive. TypeScript will complain if you forget a variant.

Step 7: Verify the validation pipeline end to end

Write a test that feeds a known-good fixture and a known-bad fixture. Use vitest and mock the fetch layer.

import { describe, it, expect, vi } from 'vitest';

vi.mock('openai', () => ({
  default: class {
    chat = {
      completions: {
        create: vi.fn().mockResolvedValue({
          choices: [{ message: { content: '{"priority":"high","category":"billing","summary":"Overcharged on invoice","tags":["refund"]}' } }],
        }),
      },
    };
  },
}));

describe('extractTicket', () => {
  it('parses valid LLM output', async () => {
    const ticket = await extractTicket('I was billed twice');
    expect(ticket.priority).toBe('high');
  });

  it('throws on malformed output', async () => {
    vi.mocked(client.chat.completions.create).mockResolvedValueOnce({
      choices: [{ message: { content: '{"priority":"urgent"}' } }],
    } as any);
    await expect(extractTicket('broken')).rejects.toThrow();
  });
});

Run npm test. If the first test passes and the second throws a ZodError, your zod llm api response validation is wired correctly. You have proof the schema rejects drift and accepts conforming JSON.

Practical caveats

Zod parsing is not free. For high-throughput paths, parse once at the boundary and pass the typed object inward. Don’t re-parse per function.

If you use streaming, accumulate the full content then validate. Validating partial JSON is a waste of cycles.

When a schema fails in production, log the issues array and the raw string. The raw string is the only evidence of what the model actually said. Without it, you are debugging blind.

Finally, version your schemas. When you change the contract, bump the version in the prompt and the schema together. A mismatch between the two is the most common source of validation noise.