LLMs still invent keys, coerce types, and silently drop fields when forced to emit JSON. The prompt patterns reduce json hallucination that hold up in production start with a contract, not a hope, and layer enforcement through the call stack. This guide walks an ordered path from schema definition to self-repair loops you can ship today.
1. Define a strict schema contract before prompting
Do not ask for “JSON with user info.” Define exact keys, types, and required fields. A JSON Schema is the most portable contract because you can reuse it for validation downstream.
{
"type": "object",
"properties": {
"user_id": { "type": "string" },
"age": { "type": "integer", "minimum": 0, "maximum": 130 },
"email": { "type": "string", "format": "email" },
"tags": { "type": "array", "items": { "type": "string" } }
},
"required": ["user_id", "email"],
"additionalProperties": false
}additionalProperties: false is the single most effective line against key hallucination. It tells the model and the validator that stray fields are invalid.
2. Anchor the schema in the system prompt
Put the schema in the system prompt, not the user message. User turns carry variable text that distracts the model; the system turn is stable context. Repeat the hard rule: output only JSON.
system_prompt = """You are a data extractor. Respond exclusively with JSON matching this schema:
{
"user_id": "string",
"age": "integer or null",
"email": "string",
"tags": ["string"]
}
Required: user_id, email. No other keys. No prose."""If you use an OpenAI-compatible client, set response_format={"type": "json_object"} as a backstop, but treat it as a hint, not a guarantee.
3. Few-shot with adversarial examples
Models mimic the shape of examples more than instructions. Show one clean extraction and one tricky case where missing data forces null instead of fabrication.
// Example 1
{"user_id":"u_1","age":29,"email":"a@b.com","tags":["vip"]}
// Example 2: note missing age, no hallucinated default
{"user_id":"u_2","age":null,"email":"c@d.com","tags":[]}Tradeoff: few-shot costs tokens on every call. For high-volume paths, cache the system prompt and examples. Gateways that forward provider cache-control hints preserve that cache across retries.
4. Command validation inside the prompt
Add a directive that forces the model to self-check before emitting. Hallucination drops when the model must explicitly acknowledge uncertainty.
Before outputting, verify: (1) all required keys present, (2) types match schema, (3) unknown values are null. Output only the JSON object.Pitfall: some models append “Here is the JSON:” or markdown fences. Strip them in post, but better to forbid in the prompt: “No markdown, no code fences.”
5. Use enums and explicit null handling
Open-ended strings are hallucination magnets. Constrain with enums wherever the domain allows.
{
"status": { "type": "string", "enum": ["active", "suspended", "unknown"] }
}If the model cannot know, unknown is a valid enum member, not a made-up value. These prompt patterns reduce json hallucination when combined with enums because the model gets a safe outlet for ignorance instead of inventing a status.
6. Separate reasoning from the JSON boundary
Asking a model to reason and emit strict JSON in one breath causes contamination: thoughts leak as extra keys. Two patterns work:
- Function calling: Define the schema as a tool and let the model call it. The JSON lives in
arguments, isolated from chain-of-thought. - Delimiter pattern: Permit free text before a marker, then JSON.
Think step by step. Then output only JSON after the line "###JSON".The delimiter pattern is cheaper but less reliable. For critical pipelines, use native function calling.
7. Test the prompt across multiple models
A prompt that works on one model often fails on another. Scripted evaluations against a fixed suite catch drift. When you route across providers, n4n.ai provides one OpenAI-compatible endpoint for 240+ models with automatic fallback, so you can pin a specific model per request via routing headers and confirm your schema holds under each backend.
curl https://api.n4n.ai/v1/chat/completions \
-H "Authorization: Bearer $KEY" \
-H "X-Route-Model: gpt-4o-mini" \
-d '{"messages":[{"role":"system","content":"..."}],"response_format":{"type":"json_object"}}'Run the same payload against three models. If one emits additionalProperties violations, either tighten the prompt or drop that model from the route.
8. Add a self-repair validation loop
No prompt is perfect. Wrap the call in code that validates and retries with the error fed back.
from pydantic import ValidationError, BaseModel
class User(BaseModel):
user_id: str
age: int | None
email: str
tags: list[str]
def extract(text: str, llm_call) -> User:
for attempt in range(3):
raw = llm_call(text)
try:
return User.model_validate_json(raw)
except ValidationError as e:
text = f"{text}\nPrevious attempt failed: {e}. Fix and output JSON only."
raise ValueError("schema not satisfied")Cap retries. Infinite repair loops burn tokens and latency for no gain. Track per-token usage metering to spot prompts that consistently need repair.
9. Tradeoffs and pitfalls to accept
- Strictness vs. recall: Rejecting invalid JSON improves quality but increases retry rate. Measure both.
- Token cost: Few-shot and self-repair multiply input tokens. Cache system prompts.
- Model variance: Smaller models hallucinate more on nested schemas. Use larger models for deep structures.
- Null abuse: Models may over-use
nullto avoid risk. Calibrate enum sets.
The prompt patterns reduce json hallucination presented here are cumulative. Schema first, enforce in system, show adversarial few-shots, command self-check, constrain enums, isolate reasoning, test broadly, then validate in code. Ship the loop, measure, and tighten.