Setting temperature and max_tokens in a curl request

Sending a chat completion with curl means getting the curl temperature max_tokens parameters right the first time, or you’ll burn tokens on runaway generations or frozen outputs. These two knobs control randomness and length, and they behave identically across OpenAI-compatible servers. This walkthrough gives you copy-paste commands and the exact response fields to inspect so you know the settings landed.

Step 1: Export credentials and pick a model

Before touching request bodies, set the key in your shell. Avoid inline secrets in command history.

export OPENAI_API_KEY="sk-..."
MODEL="gpt-4o-mini"

If you route through a gateway that aggregates providers, point BASE_URL at it. For example, n4n.ai exposes a single OpenAI-compatible endpoint covering 240+ models, so the same curl temperature max_tokens parameters work without rewriting your script.

BASE_URL="https://api.openai.com/v1"   # or https://api.n4n.ai/v1

Keep MODEL aligned with the backend you target. Some model IDs are gateway-specific aliases; others pass through to the origin provider.

Step 2: Send a minimal chat request with both parameters

The two fields live at the top level of the JSON body. temperature is a float from 0.0 to 2.0. max_tokens caps the generated tokens, not the prompt.

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"$MODEL"'",
    "messages": [{"role": "user", "content": "List three CI steps for a Go service."}],
    "temperature": 0.2,
    "max_tokens": 128
  }'

That command is the baseline. The curl temperature max_tokens parameters here ask for near-deterministic output (0.2) and a hard stop at 128 tokens. If you omit max_tokens, the server applies a default that varies by model and can surprise you with early truncation or high latency.

Step 3: Understand what temperature actually does

Temperature scales the logits before softmax. At 0.0 the model picks the argmax token every time given same prompt and state. At 1.0 you get the trained distribution. Above 1.0 flattens the distribution toward uniform, increasing nonsense risk.

For code generation or SQL, keep it ≤0.3. For brainstorming, 0.7–0.9 is sane. Never set 0 and expect creativity; you’ll get the most probable completion, which is often a short, safe phrase.

If you need reproducibility, set temperature: 0 and also seed (supported on newer models). Note that some providers ignore seed under load; gateways with fallback may route to a different backend mid-retry, breaking determinism. The curl temperature max_tokens parameters do not alone guarantee identical output across retries unless the backend is pinned.

Temperature interacts with top_p. OpenAI recommends using either temperature or top_p, not both, to avoid compounding randomness. If you set temperature: 0, leave top_p at default 1.0.

Step 4: Set max_tokens without shooting yourself in the foot

max_tokens limits completion tokens. If your prompt is 4000 tokens and you set max_tokens: 100, the total round trip is ~4100. It is not a total budget.

OpenAI’s newer models renamed the field to max_completion_tokens in their schema, but max_tokens is still accepted by the chat completions route for backward compatibility. If you see a 400 with "max_tokens": "..." it’s usually a type error (string instead of int) or exceeding the model’s absolute cap.

{
  "error": {
    "message": "'max_tokens' must be an integer",
    "type": "invalid_request_error"
  }
}

Always pass an integer. For a 128k context model, setting max_tokens: 100000 may still be rejected if the provider enforces a lower per-request completion cap. Check the model card.

To estimate tokens before sending, use a local tokenizer. For OpenAI models, tiktoken works:

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o-mini")
print(len(enc.encode("List three CI steps for a Go service.")))

Set max_tokens to leave headroom for the response. A common mistake is sizing it for the whole conversation, then watching finish_reason: "length" cut off the answer.

Step 5: Verify the parameters took effect

Success is not just a 200. Inspect the returned usage and finish_reason.

{
  "id": "chatcmpl-abc",
  "object": "chat.completion",
  "choices": [
    {
      "finish_reason": "length",
      "message": {
        "role": "assistant",
        "content": "1. go vet\n2. go test ./...\n3. go build"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 128,
    "total_tokens": 140
  }
}

If finish_reason is "length", you hit max_tokens. If it’s "stop", the model ended naturally before the cap. The completion_tokens should be ≤ your setting. If you sent temperature: 0 and repeat the call with same seed and get identical content, the curl temperature max_tokens parameters are wired correctly.

A quick verification script:

for i in 1 2; do
  curl -s "$BASE_URL/chat/completions" \
    -H "Authorization: Be $OPENAI_API_KEY" \
    -d '{"model":"'"$MODEL"'","messages":[{"role":"user","content":"Say hi."}],"temperature":0,"max_tokens":5}' \
    | python3 -c "import sys,json; print(json.load(sys.stdin)['choices'][0]['message']['content'])"
done

Two identical prints confirm deterministic behavior. Extract usage with jq to assert the cap:

curl -s "$BASE_URL/chat/completions" \
  -d '{"model":"'"$MODEL"'","messages":[{"role":"user","content":"Hi"}],"temperature":0.5,"max_tokens":10}' \
  | jq '.usage.completion_tokens, .choices[0].finish_reason'

Step 6: Streaming and JSON mode with the same parameters

Streaming doesn’t change parameter semantics. You still pass stream: true alongside them.

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "'"$MODEL"'",
    "messages": [{"role":"user","content":"Return a JSON list of 2 build steps."}],
    "temperature": 0.1,
    "max_tokens": 64,
    "stream": true,
    "response_format": {"type": "json_object"}
  }'

With response_format, the model is constrained to valid JSON, but max_tokens can truncate mid-object. Keep the cap generous enough for the schema. When streaming, finish_reason appears in the final SSE chunk with value length or stop, same as non-streaming.

If you parse streams, accumulate chunks and check the last finish_reason. Do not assume the first chunk carries it.

Step 7: Route-aware settings and gateway fallbacks

If your BASE_URL is a gateway, the curl temperature max_tokens parameters are forwarded verbatim. A gateway that honors client routing directives may let you pin a provider via headers:

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "X-Route: provider=anthropic" \
  -d '{"model":"claude-3-5-sonnet","messages":[{"role":"user","content":"Hi"}],"temperature":0.5,"max_tokens":32}'

When the pinned provider is degraded, automatic fallback switches backend; your temperature and max_tokens still apply on the retry. Per-token usage metering in the response reflects the actual backend used. n4n.ai forwards provider cache-control hints, so a temperature=0 request with an identical prompt can hit a provider cache and return instantly without recomputation.

Step 8: Common failures and how to debug

400 invalid_request_error – Usually a string where an int was expected, or temperature outside [0,2]. Validate with jq before sending:

echo '{"temperature":0.2,"max_tokens":128}' | jq '.max_tokens | type'
# outputs "number"

Truncated output but finish_reason is stop – You likely set max_tokens too low. Bump it.

Non-deterministic at temperature 0 – Some models have a tiny inherent noise; add seed and ensure the gateway didn’t route to a different model variant.

Rate limits – 429s are unrelated to parameters. Back off and retry; gateways with fallback handle this transparently.

JSON mode returns incomplete JSON – max_tokens cut the serialization. Raise the cap or simplify the requested schema.

Step 9: Put it in a reusable shell function

Stop rewriting the body. Wrap it:

chat() {
  local prompt="$1"
  local temp="${2:-0.3}"
  local maxtok="${3:-256}"
  curl -s "$BASE_URL/chat/completions" \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -d '{
      "model": "'"$MODEL"'",
      "messages": [{"role":"user","content": "'"$prompt"'"}],
      "temperature": '"$temp"',
      "max_tokens": '"$maxtok"'
    }' | jq -r '.choices[0].message.content'
}

Call chat "Explain TCP fast open" 0.3 256 and you get trimmed output. Change the defaults in one place. This pattern keeps the curl temperature max_tokens parameters consistent across every invocation.

Step 10: Parameter precedence in multi-turn contexts

When you send multiple messages, temperature and max_tokens apply to the completion, not per message. The prior assistant messages are fixed; only the generated tail respects the cap. If you maintain a rolling buffer, count tokens in the whole messages array with a tokenizer and subtract from the model context limit to choose a safe max_tokens.

A misconfigured max_tokens in a loop will silently truncate each reply, causing the next turn to lose context. Log usage.completion_tokens each call to catch drift.

Final checks

You now have runnable curl commands that set temperature and max_tokens correctly, verify the response, and degrade gracefully. The curl temperature max_tokens parameters are the foundation for every downstream tuning trick—caching, seed locking, and provider routing all assume these two are correct first. Test with temperature: 0 for determinism, watch finish_reason, and never trust a 200 without inspecting usage.