How to structure a prompt for consistent outputs

Most engineers treat prompting as trial and error. That works for prototypes. It fails in production where you need the same schema, tone, and reasoning every time. Learning how to structure a prompt for consistency means treating the prompt like code: versioned, tested, and bounded by explicit contracts. Below is the exact process I use to take a fuzzy requirement to a prompt that passes automated regression checks.

Step 1: Define the output contract first

Before writing a single instruction, write the JSON Schema (or TypeScript interface) that the model must satisfy. This is your acceptance criteria. If you cannot express the valid output space as a schema, you do not yet understand the task well enough to prompt it.

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["decision", "confidence", "reasoning"],
  "properties": {
    "decision": { "type": "string", "enum": ["approve", "reject", "escalate"] },
    "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
    "reasoning": { "type": "string", "maxLength": 500 }
  },
  "additionalProperties": false
}

Save this as schemas/loan-decision.json. Every prompt iteration must produce output that validates against it. No exceptions.

Step 2: Decompose the task into deterministic subtasks

LLMs fail when a single prompt tries to do classification, extraction, calculation, and formatting simultaneously. Break the work into a chain where each step has a single, verifiable responsibility.

For a loan application review, the chain might be:

1. Extract structured fields from raw text (income, debt, employment history)
2. Calculate ratios (DTI, LTV) — do this in code, not the model
3. Classify risk tier using the extracted + calculated fields
4. Format final decision per the schema from Step 1

Each step gets its own prompt template. This isolates failure modes and lets you swap models per step (e.g., a small fast model for extraction, a larger one for classification).

Step 3: Write the system prompt as a strict specification

The system prompt is not a personality sketch. It is a machine-readable spec. Include:

• Role and authority boundaries
• Output format (reference the schema file)
• Hard constraints (no markdown, no commentary, specific enum values)
• Few-shot examples covering edge cases

You are a loan underwriting engine. Output ONLY a JSON object that validates against the schema at schemas/loan-decision.json.

Constraints:
- decision must be exactly one of: approve, reject, escalate
- confidence is a float 0.0–1.0 with two decimal places
- reasoning is plain text, ≤500 chars, no markdown
- If any required input field is missing, set decision="escalate" and confidence=0.00

Examples:
Input: {"income": 120000, "debt": 30000, "employment_years": 5, "loan_amount": 200000, "property_value": 250000}
Output: {"decision": "approve", "confidence": 0.92, "reasoning": "DTI 25%, LTV 80%, stable employment. Meets all thresholds."}

Input: {"income": 45000, "debt": 40000, "employment_years": 1, "loan_amount": 300000, "property_value": 320000}
Output: {"decision": "reject", "confidence": 0.97, "reasoning": "DTI 89% exceeds 43% limit. Employment <2 years."}

Input: {"income": 80000, "debt": 20000, "employment_years": 3, "loan_amount": 400000, "property_value": 450000}
Output: {"decision": "escalate", "confidence": 0.00, "reasoning": "LTV 89% near threshold. Requires manual review per policy."}

Store this as prompts/underwrite-system.txt. Version it like code.

Step 4: Build the user prompt template with typed slots

The user prompt carries the variable data. Use a template engine (Jinja2, Python f-strings, Go text/template) — never string concatenation. This prevents injection and ensures consistent formatting.

# prompts/underwrite_user.j2
Analyze the following loan application:

Applicant:
- Annual income: ${{ income }}
- Monthly debt payments: ${{ debt }}
- Years at current employer: {{ employment_years }}

Loan:
- Requested amount: ${{ loan_amount }}
- Property appraised value: ${{ property_value }}

Calculated metrics (pre-computed):
- Debt-to-income ratio: {{ dti_pct }}%
- Loan-to-value ratio: {{ ltv_pct }}%

Respond with ONLY the JSON decision object.

Render it in your application code:

from jinja2 import Environment, FileSystemLoader
import json

env = Environment(loader=FileSystemLoader("prompts"))
template = env.get_template("underwrite_user.j2")

def build_user_prompt(app: dict) -> str:
    dti = (app["debt"] * 12 / app["income"]) * 100
    ltv = (app["loan_amount"] / app["property_value"]) * 100
    return template.render(
        income=app["income"],
        debt=app["debt"],
        employment_years=app["employment_years"],
        loan_amount=app["loan_amount"],
        property_value=app["property_value"],
        dti_pct=round(dti, 1),
        ltv_pct=round(ltv, 1),
    )

Step 5: Enforce the schema at inference time

Do not trust the model to follow the schema. Validate every response. If validation fails, retry with a correction prompt (max 2 retries), then escalate to a dead-letter queue for human review.

import jsonschema
from jsonschema import validate
from openai import OpenAI

client = OpenAI()  # or your gateway client
SCHEMA = json.load(open("schemas/loan-decision.json"))

CORRECTION_PROMPT = """Your previous response failed schema validation.
Error: {{ error }}
Schema: {{ schema }}
Fix the output. Return ONLY valid JSON."""

def call_with_validation(system_prompt: str, user_prompt: str, max_retries: int = 2) -> dict:
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt},
    ]

    for attempt in range(max_retries + 1):
        resp = client.chat.completions.create(
            model="gpt-4o-mini",  # or your routed model
            messages=messages,
            temperature=0,
            response_format={"type": "json_object"},
        )
        raw = resp.choices[0].message.content
        try:
            data = json.loads(raw)
            validate(instance=data, schema=SCHEMA)
            return data
        except (json.JSONDecodeError, jsonschema.ValidationError) as e:
            if attempt == max_retries:
                raise
            # Retry with correction
            correction = CORRECTION_PROMPT.replace("{{ error }}", str(e)).replace("{{ schema }}", json.dumps(SCHEMA))
            messages.append({"role": "assistant", "content": raw})
            messages.append({"role": "user", "content": correction})

    raise RuntimeError("Exhausted retries")

Setting temperature=0 and response_format={"type": "json_object"} (or the equivalent for your provider) eliminates two major sources of variance.

Step 6: Pin the model and sampling parameters

Consistency requires a fixed model identifier and frozen sampling config. Do not use aliases like “gpt-4o” that can resolve to different snapshots. Pin to a dated snapshot or a specific provider model ID.

MODEL_CONFIG = {
    "model": "gpt-4o-mini-2024-07-18",  # dated snapshot
    "temperature": 0,
    "top_p": 1,
    "max_tokens": 500,
    "response_format": {"type": "json_object"},
}

If you route across providers (e.g., via n4n.ai), pin the routing directive in your request headers so the same logical model is selected every time:

headers = {
    "X-Model": "gpt-4o-mini-2024-07-18",
    "X-Fallback": "false",  # disable automatic fallback for reproducibility
}

Step 7: Create a golden test set and run it in CI

Collect 30–50 representative inputs covering happy paths, boundary values, and known failure modes. Store them as JSONL with expected outputs.

{"input": {"income": 120000, "debt": 30000, "employment_years": 5, "loan_amount": 200000, "property_value": 250000}, "expected": {"decision": "approve", "confidence": 0.92}}
{"input": {"income": 45000, "debt": 40000, "employment_years": 1, "loan_amount": 300000, "property_value": 320000}, "expected": {"decision": "reject", "confidence": 0.97}}
{"input": {"income": 80000, "debt": 20000, "employment_years": 3, "loan_amount": 400000, "property_value": 450000}, "expected": {"decision": "escalate", "confidence": 0.00}}

Run the test suite on every prompt change:

# tests/test_prompt_consistency.py
import pytest
from app.prompt_runner import run_underwrite_prompt

CASES = [json.loads(line) for line in open("tests/golden/loan_cases.jsonl")]

@pytest.mark.parametrize("case", CASES)
def test_golden_case(case):
    result = run_underwrite_prompt(case["input"])
    # Exact match on decision, tolerance on confidence
    assert result["decision"] == case["expected"]["decision"]
    assert abs(result["confidence"] - case["expected"]["confidence"]) < 0.05

Add this to your CI pipeline. A failing test means the prompt (or model) drifted — roll back or update the golden set intentionally.

Step 8: Log everything for post-hoc analysis

Structured logs let you catch silent degradation before users do. Log the request ID, model snapshot, full prompt, raw response, parsed output, validation result, and latency.

import structlog
import uuid

logger = structlog.get_logger()

def run_underwrite_prompt(app: dict) -> dict:
    request_id = str(uuid.uuid4())
    user_prompt = build_user_prompt(app)
    system_prompt = open("prompts/underwrite-system.txt").read()

    logger.info("prompt_request", request_id=request_id, model=MODEL_CONFIG["model"], input=app)

    try:
        result = call_with_validation(system_prompt, user_prompt)
        logger.info("prompt_success", request_id=request_id, output=result)
        return result
    except Exception as e:
        logger.error("prompt_failure", request_id=request_id, error=str(e))
        raise

Query these logs weekly: group by decision, check confidence distributions, flag any escalate rate spikes.

Step 9: Version prompts alongside model upgrades

When a new model snapshot is released, treat it as a dependency upgrade. Create a branch, update MODEL_CONFIG["model"], run the golden test suite. If tests pass, merge. If they fail, you have two choices:

1. Adjust the prompt (add/remove examples, tighten constraints) until tests pass
2. Keep the old model pinned and file a ticket to investigate

Never silently upgrade. The golden test suite is your contract test — it tells you whether the new model honors the same specification.

Verification checklist

Before merging any prompt change, confirm:

• Schema file exists and validates the expected output shape
• System prompt references the schema by path
• User prompt uses a template engine, no string interpolation
• temperature=0 and response_format=json_object are set
• Model identifier is a dated snapshot, not an alias
• Golden test suite passes locally
• CI runs the golden suite on every PR
• Structured logs include request ID, model, input, output, latency
• Fallback routing is disabled for this prompt (determinism > availability)

Common failure patterns to avoid

When to break these rules

The only exception: exploratory tasks where you genuinely do not know the output space yet (e.g., “summarize this unknown document”). For those, use a separate prompt pipeline with looser validation, human-in-the-loop review, and a clear path to formalize once patterns emerge. Do not mix exploratory and production prompts in the same code path.

Consistency comes from constraints you can test. Define the contract, decompose the work, pin the model, validate every response, and run golden tests in CI. That is how to structure a prompt that behaves like reliable infrastructure instead of a demo.