n4nAI
Open console →
GlossaryGuideDeterminism, Seeds & Reproducibility in LLMs

How OpenAI's seed parameter works, and its limits

Understand how OpenAI's seed parameter enables reproducibility, where it falls short, and practical patterns for deterministic LLM outputs in production.

n4n Team4 min read875 words

Audio narration

Coming soon — every post will get a voice note here.

The openai seed parameter limits what you can reliably reproduce across requests. It gives you a stable starting point for the sampling process, but it does not guarantee identical outputs when model weights, system fingerprints, or provider infrastructure change. This guide walks through how the parameter works, where the guarantees break down, and the patterns that actually work in production.

What the seed parameter actually controls

When you pass seed in a chat completion request, OpenAI uses it to initialize the random number generator that drives token sampling. The same seed with the same prompt, same model, same temperature, and same system fingerprint should produce the same token sequence.

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a haiku about debugging"}],
    temperature=0.7,
    seed=42,
    max_tokens=100
)

print(response.choices[0].message.content)
print("System fingerprint:", response.system_fingerprint)

The system_fingerprint field in the response is critical. It encodes the model version and inference configuration that produced the output. If that fingerprint changes, the same seed will likely produce different results.

The reproducibility contract: what’s promised vs. what’s delivered

OpenAI documents that the seed parameter “makes a best effort to sample deterministically.” The key phrase is “best effort.” The contract covers:

  • Identical prompt + identical parameters + identical model version + identical system fingerprint = identical output
  • Temperature must be > 0 for seed to have any effect (at temperature 0, sampling is already deterministic)
  • The seed is a 32-bit integer (0 to 2^31-1)

What’s not covered:

  • Model updates (even minor ones) invalidate reproducibility
  • Infrastructure changes across regions or capacity pools can shift behavior
  • The system fingerprint may change without notice
  • No SLA or version pinning for the fingerprint itself

Verifying determinism in your CI pipeline

Don’t assume determinism works. Test it. Here’s a pattern that catches regressions:

import hashlib
import json
from openai import OpenAI

client = OpenAI()

def get_deterministic_output(seed: int, prompt: str, model: str = "gpt-4o-mini") -> tuple[str, str]:
    """Returns (content_hash, system_fingerprint) for a given seed."""
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7,
        seed=seed,
        max_tokens=200
    )
    content = response.choices[0].message.content
    content_hash = hashlib.sha256(content.encode()).hexdigest()[:16]
    return content_hash, response.system_fingerprint

# Test reproducibility
prompt = "Explain the CAP theorem in two sentences."
seed = 12345

hash1, fp1 = get_deterministic_output(seed, prompt)
hash2, fp2 = get_deterministic_output(seed, prompt)

print(f"Run 1: {hash1} (fp: {fp1})")
print(f"Run 2: {hash2} (fp: {fp2})")
print(f"Deterministic: {hash1 == hash2 and fp1 == fp2}")

Run this in CI weekly. If the hash or fingerprint changes, your deterministic workflows need attention.

Common pitfalls that break reproducibility

Temperature and top_p interaction

Setting temperature=0 makes the seed irrelevant — the model always picks the highest-probability token. But temperature=0.0001 with a seed is not the same as temperature=0. The tiny randomness still routes through the seeded RNG.

# These produce DIFFERENT results despite same seed
response_a = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to 5"}],
    temperature=0,
    seed=42
)

response_b = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to 5"}],
    temperature=0.0001,
    seed=42
)

Message formatting differences

Whitespace, message ordering, and system prompt variations all change the prompt hash internally. Two prompts that look identical to you may tokenize differently.

# These are NOT the same prompt to the model
prompt_a = "Hello"
prompt_b = "Hello "  # trailing space
prompt_c = "\nHello"  # leading newline

Normalize inputs before sending. Strip whitespace, enforce consistent message ordering, and pin system prompts.

Streaming breaks seed guarantees

When stream=True, the seed parameter is accepted but the determinism guarantee does not hold across stream chunks. The sampling state can diverge mid-stream due to chunking boundaries and buffering.

# Don't rely on this for reproducibility
stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a poem"}],
    temperature=0.7,
    seed=42,
    stream=True
)

# Collect and compare — will often differ from non-streaming
chunks = []
for chunk in stream:
    if chunk.choices[0].delta.content:
        chunks.append(chunk.choices[0].delta.content)
streamed_output = "".join(chunks)

If you need deterministic streaming, collect the full non-streamed response first, then simulate streaming on your side.

Practical patterns for production determinism

Pin the model version explicitly

Never use aliases like gpt-4o or gpt-4o-mini in production if you need reproducibility. Use dated snapshots:

# Good: explicit version
MODEL = "gpt-4o-mini-2024-07-18"

# Bad: rolling alias
MODEL = "gpt-4o-mini"

Check the model deprecation schedule. Dated snapshots eventually retire. Build a model rotation process that re-validates your deterministic workflows.

Store the system fingerprint with every completion

Treat the system fingerprint as part of your output contract. Log it, store it, and compare it on replay.

import json
from dataclasses import dataclass, asdict
from datetime import datetime

@dataclass
class CompletionRecord:
    prompt_hash: str
    seed: int
    temperature: float
    model: str
    system_fingerprint: str
    output_hash: str
    timestamp: str
    output_text: str

def record_completion(prompt: str, seed: int, temperature: float, model: str, response) -> CompletionRecord:
    import hashlib
    prompt_hash = hashlib.sha256(prompt.encode()).hexdigest()[:16]
    output_text = response.choices[0].message.content
    output_hash = hashlib.sha256(output_text.encode()).hexdigest()[:16]
    
    return CompletionRecord(
        prompt_hash=prompt_hash,
        seed=seed,
        temperature=temperature,
        model=model,
        system_fingerprint=response.system_fingerprint,
        output_hash=output_hash,
        timestamp=datetime.utcnow().isoformat(),
        output_text=output_text
    )

Replay verification endpoint

Build a /replay endpoint that takes a stored record, re-sends the request, and verifies the output matches. This catches fingerprint drift automatically.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class ReplayRequest(BaseModel):
    prompt: str
    seed: int
    temperature: float
    model: str
    expected_fingerprint: str
    expected_output_hash: str

@app.post("/replay")
async def replay(req: ReplayRequest):
    response = client.chat.completions.create(
        model=req.model,
        messages=[{"role": "user", "content": req.prompt}],
        temperature=req.temperature,
        seed=req.seed,
        max_tokens=500
    )
    
    actual_fingerprint = response.system_fingerprint
    actual_output = response.choices[0].message.content
    actual_hash = hashlib.sha256(actual_output.encode()).hexdigest()[:16]
    
    if actual_fingerprint != req.expected_fingerprint:
        raise HTTPException(
            status_code=409,
            detail=f"Fingerprint drift: expected {req.expected_fingerprint}, got {actual_fingerprint}"
        )
    
    if actual_hash != req.expected_output_hash:
        raise HTTPException(
            status_code=409,
            detail=f"Output divergence: expected {req.expected_output_hash}, got {actual_hash}"
        )
    
    return {"status": "verified", "output": actual_output}

Use temperature 0 for true determinism when possible

If your use case tolerates greedy decoding, temperature=0 is more reliable than any seed. It eliminates the sampling layer entirely.

# Most deterministic option — no seed needed
response = client.chat.completions.create(
    model="gpt-4o-mini-2024-07-18",
    messages=[{"role": "user", "content": "Extract the JSON from this text: ..."}],
    temperature=0,
    max_tokens=500
)

Tradeoff: temperature 0 can produce repetitive or stuck outputs on open-ended tasks. Reserve it for extraction, classification, and structured generation.

When the seed parameter is the wrong tool

Cross-provider reproducibility

The seed parameter is OpenAI-specific. Anthropic, Google, and open-source models have their own seed implementations (or none at all). If you route across providers, seed-based determinism won’t transfer.

If you’re using a gateway that normalizes across providers — like n4n.ai’s single endpoint addressing 240+ models — you’ll need provider-specific determinism strategies. The gateway forwards provider cache-control hints and honors routing directives, but each upstream has its own reproducibility contract.

Long-running conversations

In multi-turn conversations, the seed applies to each completion independently. The conversation history grows, changing the prompt each turn. You cannot “resume” a deterministic conversation from a checkpoint by reusing the seed.

# This doesn't work as a "resume" mechanism
turn_1 = client.chat.completions.create(
    model=MODEL, messages=[{"role": "user", "content": "Start a story"}],
    temperature=0.7, seed=42
)

# Turn 2 with same seed ≠ continuation of turn 1
turn_2 = client.chat.completions.create(
    model=MODEL, 
    messages=[
        {"role": "user", "content": "Start a story"},
        {"role": "assistant", "content": turn_1.choices[0].message.content},
        {"role": "user", "content": "Continue"}
    ],
    temperature=0.7, seed=42  # Different prompt = different output
)

For conversation determinism, you must replay the entire history from turn 1 with the same seed each time.

Evaluation and benchmarking

For evals, prefer temperature 0 with multiple fixed prompts over seeded sampling. Seeded sampling at temperature > 0 introduces variance that’s hard to control and harder to explain in results.

# Better eval pattern: multiple prompts, temperature 0
eval_prompts = [
    "Classify: 'I love this product'",
    "Classify: 'This is terrible'",
    "Classify: 'It's okay I guess'"
]

results = []
for prompt in eval_prompts:
    response = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
        temperature=0
    )
    results.append(response.choices[0].message.content)

Monitoring for fingerprint drift

Set up alerting on system fingerprint changes. A simple cron job:

#!/bin/bash
# check_fingerprint.sh - run daily via cron

MODEL="gpt-4o-mini-2024-07-18"
PROMPT="Test prompt for fingerprint monitoring"
SEED=99999

RESPONSE=$(curl -s https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"$PROMPT\"}],\"temperature\":0.7,\"seed\":$SEED,\"max_tokens\":10}")

FINGERPRINT=$(echo $RESPONSE | jq -r '.system_fingerprint')
STORED_FINGERPRINT=$(cat /etc/llm_fingerprint 2>/dev/null || echo "unknown")

if [ "$FINGERPRINT" != "$STORED_FINGERPRINT" ]; then
  echo "FINGERPRINT_CHANGED: $STORED_FINGERPRINT -> $FINGERPRINT" | \
    mail -s "LLM Fingerprint Drift Alert" oncall@yourcompany.com
  echo $FINGERPRINT > /etc/llm_fingerprint
fi

Summary checklist

Before relying on the seed parameter in production:

  • Pin to a dated model snapshot (e.g., gpt-4o-mini-2024-07-18)
  • Use temperature > 0 only when you need sampling diversity
  • Normalize prompts: strip whitespace, fix message ordering, pin system prompts
  • Log system_fingerprint with every completion
  • Build replay verification into your deployment pipeline
  • Monitor for fingerprint drift with automated alerts
  • Accept that model updates will break reproducibility — plan for re-validation
  • Don’t use seed for cross-provider or cross-model determinism
  • Don’t use streaming if you need guaranteed reproducibility

The seed parameter is a useful tool, not a guarantee. Treat it like a cache key: valid until the underlying system changes, then invalidate and rebuild.

Tagsseed-parameteropenai-apideterminism

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
GlossaryWhat is the seed parameter in LLM APIs?A technical explainer of the seed parameter in LLM APIs — what it controls, how it enables reproducibility, and the practical limits engineers should know.GlossaryDeterminism in production: what you can actually controlA practical guide to achieving deterministic LLM outputs in production — covering seeds, temperature, prompt locking, provider quirks, and fallback strategies.GlossaryHow to test LLM outputs when results aren't reproducibleA practical guide to testing non-reproducible LLM outputs using deterministic sampling, statistical evaluation, property-based checks, and production drift monitoring.GlossaryWhy exact reproducibility is hard to get from an LLM APIAn analysis of why LLM APIs cannot offer true reproducibility guarantees, covering hardware non-determinism, provider infrastructure, and practical mitigation strategies.GlossaryWhy the same prompt can give different answers twiceUnderstand why identical LLM prompts produce different outputs — sampling, seeds, floating-point non-determinism, and practical reproducibility strategies.GlossaryHow OpenAI's frequency_penalty parameter actually worksA practical guide to OpenAI's frequency_penalty parameter — how the math works, when to apply it, tuning strategies, and common mistakes that waste tokens.

All determinism, seeds & reproducibility in llms posts →