n4nAI
Open console →
GlossaryHow-toMax Tokens, Stop Sequences & Output Truncation

How stop sequences work in the OpenAI and Claude APIs

Learn how stop sequences control LLM output termination in OpenAI and Claude APIs with practical code examples and verification steps.

n4n Team4 min read864 words

Audio narration

Coming soon — every post will get a voice note here.

Stop sequences are the most reliable way to bound LLM output without truncating mid-token. Both OpenAI and Claude accept an array of strings that, when generated, immediately halt completion — the stop sequence itself is excluded from the response. This guide walks through configuring them correctly, handling edge cases, and verifying behavior across providers.

Step 1: Understand the stop sequence contract

A stop sequence is a string (or array of strings) that tells the model “stop generating when you emit this exact sequence.” The sequence is not included in the returned text. Both APIs treat stop sequences as case-sensitive exact matches — “STOP” does not match “stop” or “Stop”.

OpenAI allows up to 4 stop sequences per request. Claude allows up to 5. Neither guarantees the model won’t generate the sequence as part of normal output before the intended stopping point; they only guarantee termination when the sequence appears.

# Conceptual: what the API does internally
def apply_stop_sequences(text: str, stop_sequences: list[str]) -> str:
    for seq in stop_sequences:
        idx = text.find(seq)
        if idx != -1:
            return text[:idx]
    return text

Step 2: Configure stop sequences in OpenAI

Pass the stop parameter in your chat completion request. It accepts a string or array of strings.

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a SQL generator. Output only the query."},
        {"role": "user", "content": "List all users created in the last 30 days."}
    ],
    stop=[";", "```"],  # Stop at semicolon or code fence close
    max_tokens=200,
    temperature=0
)

sql = response.choices[0].message.content
print(repr(sql))  # Verify: no trailing semicolon, no backticks

Verification: Run the request with temperature=0 and inspect repr(output). The output should end immediately before your stop sequence with no extra whitespace. If you see the stop sequence in the output, the model generated it before the API could intercept — this happens when the sequence appears mid-generation rather than as a deliberate terminator.

Step 3: Configure stop sequences in Claude

Claude uses the stop_sequences parameter (plural, snake_case) in the Messages API.

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=200,
    temperature=0,
    system="You are a SQL generator. Output only the query.",
    messages=[
        {"role": "user", "content": "List all users created in the last 30 days."}
    ],
    stop_sequences=[";", "```"]
)

sql = response.content[0].text
print(repr(sql))

Verification: Same as OpenAI — check repr(output) for clean termination. Claude’s stop_reason field in the response will be "stop_sequence" when a stop sequence triggered the end, versus "max_tokens" or "end_turn" for other termination reasons.

# Check why generation stopped
print(response.stop_reason)  # "stop_sequence" | "max_tokens" | "end_turn" | "tool_use"
print(response.stop_sequence)  # The specific sequence that matched, if any

Step 4: Handle multi-sequence priority

When multiple stop sequences are provided, both APIs stop at the first match in the generated text, not the first in your array. The model generates token by token; whichever sequence appears first in the stream wins.

# If model outputs: "SELECT * FROM users;\n```"
# stop=[";", "```"]  -> stops at semicolon (appears first)
# stop=["```", ";"]  -> still stops at semicolon (appears first in output)

Order in your array only matters for the stop_sequence field returned by Claude (it reports which sequence matched). OpenAI does not report which sequence triggered.

Step 5: Use stop sequences for structured output

Stop sequences excel at extracting single fields or enforcing format boundaries without parsing.

Extract a single JSON field

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Output only the JSON value for the requested key."},
        {"role": "user", "content": 'Extract "email" from: {"name": "Alice", "email": "alice@example.com", "role": "admin"}'}
    ],
    stop=["}", ","],  # Stop at object end or next field
    max_tokens=50,
    temperature=0
)
# Output: "alice@example.com"

Enforce code block boundaries

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Write a Python function. Wrap in ```python```."},
        {"role": "user", "content": "Function to compute fibonacci(n)"}
    ],
    stop=["```"],  # Stops at closing fence
    max_tokens=300,
    temperature=0.2
)
# Output includes opening ```python but stops before closing ```

Verification: Parse the output as JSON or extract the code block programmatically. If parsing fails, the stop sequence likely didn’t fire — check for model hallucination of the sequence inside the content.

Step 6: Combine with max_tokens for safety

Stop sequences are not a substitute for max_tokens. If the model never generates your stop sequence, generation continues until max_tokens or the model’s natural end. Always set both.

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a very long essay."}],
    stop=["CONCLUSION:"],
    max_tokens=500,  # Hard ceiling
    temperature=0.7
)

Verification: Check response.usage.completion_tokens. If it equals max_tokens, the stop sequence never fired. Log this metric in production to detect prompt drift.

Step 7: Handle streaming responses

In streaming mode, stop sequences work identically — the stream terminates when the sequence is detected. However, the final chunk may contain partial content before the stop sequence.

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to 10."}],
    stop=["5"],
    max_tokens=50,
    stream=True
)

collected = []
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    collected.append(delta)
    print(delta, end="", flush=True)

full = "".join(collected)
print(f"\n\nFinal: {repr(full)}")
# Output: "1, 2, 3, 4, " (stops before "5")

Verification: The final accumulated string should not contain the stop sequence. In streaming, you cannot rely on finish_reason in intermediate chunks — only the final chunk carries it.

Step 8: Avoid common pitfalls

Whitespace sensitivity

Stop sequences match exactly. "\n" matches a newline; " \n" matches space-then-newline. Models often emit inconsistent whitespace.

# Robust: include common variations
stop=["\n", "\n\n", " \n", "\n ", "END"]

Tokenization boundaries

A stop sequence split across tokens may not match. For example, if “STOP” tokenizes as ["ST", "OP"] and the model generates “ST” then “OP”, the API still detects “STOP” in the decoded text and stops. But if your stop sequence is “STOP\n” and the model generates “STOP” then “\n” as separate tokens, it still matches — the API operates on decoded text, not token IDs.

However, byte-level mismatches can occur with Unicode. Prefer ASCII stop sequences.

Model ignoring stop sequences

At high temperatures, models may generate the stop sequence as content rather than a terminator. The API still stops at the first occurrence.

# At temperature=1.0, model might write: "The answer is STOP right here."
# With stop=["STOP"], output: "The answer is "

Fix: Lower temperature, use more unique sequences ("###END###"), or post-process.

Step 9: Test stop sequence behavior systematically

Add a test harness to your CI that validates stop sequences against your actual prompts.

import pytest
from openai import OpenAI

client = OpenAI()

STOP_TEST_CASES = [
    {
        "name": "sql_semicolon",
        "messages": [
            {"role": "system", "content": "Output only SQL."},
            {"role": "user", "content": "Select all users"}
        ],
        "stop": [";"],
        "assert": lambda out: not out.endswith(";") and ";" not in out
    },
    {
        "name": "json_field_extraction",
        "messages": [
            {"role": "system", "content": "Extract only the email value."},
            {"role": "user", "content": '{"email": "test@example.com", "name": "Test"}'}
        ],
        "stop": ["}", ","],
        "assert": lambda out: out == "test@example.com"
    }
]

@pytest.mark.parametrize("case", STOP_TEST_CASES)
def test_stop_sequences(case):
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=case["messages"],
        stop=case["stop"],
        max_tokens=100,
        temperature=0
    )
    output = resp.choices[0].message.content
    assert case["assert"](output), f"{case['name']}: got {repr(output)}"

Run this periodically. Model updates can change formatting behavior silently.

Step 10: Route stop sequences through a gateway

If you’re calling multiple providers, normalize the parameter name and validation in your gateway layer. The OpenAI-compatible endpoint at n4n.ai accepts both stop and stop_sequences, forwards them to the upstream provider correctly, and returns the provider’s stop_reason in the response metadata so you can audit termination causes across models without provider-specific parsing.

# Gateway-normalized request
response = gateway.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[...],
    stop=[";", "```"],  # Works for both OpenAI and Claude upstreams
    max_tokens=200
)

# Unified response inspection
print(response.choices[0].finish_reason)  # "stop" for both providers
print(response.metadata.get("upstream_stop_reason"))  # "stop_sequence" | "max_tokens" | etc.

Summary checklist

  • Use stop (OpenAI) or stop_sequences (Claude) with an array of strings
  • Set max_tokens as a hard ceiling — stop sequences are not guaranteed to fire
  • Test with temperature=0 for deterministic verification
  • Inspect finish_reason / stop_reason to confirm termination cause
  • Include whitespace variations in your stop sequences for robustness
  • Prefer unique, ASCII-only sequences unlikely to appear in content
  • Log completion_tokens == max_tokens as a “stop sequence missed” metric
  • Add automated tests for critical prompt/stop-sequence pairs

Stop sequences are a precise tool. Treat them like regex anchors — exact, brittle, and invaluable when you control the grammar.

Tagsstop-sequencesopenai-apiclaude-api

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
GlossaryCommon max_tokens mistakes that truncate JSON outputSeven max_tokens mistakes that silently truncate JSON output, with fixes for token budgeting, reasoning models, streaming, and provider quirks.GlossaryHow to detect and handle truncated LLM responsesLearn to detect truncated LLM responses using finish_reason and usage fields, then implement retries, continuation prompts, and monitoring to handle incomplete outputs reliably.GlossaryHow to set max_tokens for long-form content generationA step-by-step guide to configuring max_tokens for long-form content generation, covering model limits, chunking strategies, and verification techniques.GlossaryMax output tokens by model: GPT-4o, Claude, and GeminiCompare max output tokens across GPT-4o, Claude, and Gemini with practical guidance for handling truncation and stop sequences in production.GlossaryMax tokens explained: input, output, and total limitsUnderstand how max tokens, input limits, and output budgets interact across providers — with code patterns for safe truncation, streaming, and cost control.GlossaryWhat does max_tokens actually control in an API call?A precise technical explanation of what max_tokens controls in LLM API calls, how it interacts with context windows and stop sequences, and common misconceptions that cause production bugs.

All max tokens, stop sequences & output truncation posts →