When you call an LLM API without specifying sampling parameters, you get the provider’s default sampling settings by model. Those defaults shape every token your application generates, yet they differ significantly across GPT-4o, Claude, and Llama 3. Understanding the gaps helps you decide when to accept the defaults and when to override them.
What the APIs actually expose
OpenAI, Anthropic, and the various Llama 3 hosts each surface a different subset of sampling knobs. OpenAI offers temperature, top_p, frequency_penalty, and presence_penalty. Anthropic adds top_k to that list. Llama 3 deployments — whether through Ollama, vLLM, TGI, or managed endpoints like Together or Fireworks — typically expose all four plus min_p, typical_p, and sometimes mirostat. The parameter surface area alone changes how you tune.
Temperature defaults
| Model | Default temperature | Range |
|---|---|---|
| GPT-4o | 1.0 | 0–2 |
| Claude 3 Opus/Sonnet/Haiku | 1.0 | 0–1 |
| Llama 3 (Meta reference) | 0.6 | 0–2 |
| Llama 3 (Ollama default) | 0.8 | 0–2 |
| Llama 3 (vLLM/TGI default) | 1.0 | 0–2 |
GPT-4o and Claude both ship with temperature 1.0, meaning full entropy from the logits. Meta’s own Llama 3 documentation suggests 0.6 for chat use cases, but most inference servers default to 0.8 or 1.0. If you move a prompt from OpenAI to a self-hosted Llama 3 endpoint without setting temperature explicitly, you’ll likely see more variance than you expect.
Top-p and top-k defaults
| Model | Default top_p | Default top_k |
|---|---|---|
| GPT-4o | 1.0 | Not exposed |
| Claude 3 | 0.999 | 250 |
| Llama 3 (Meta) | 0.9 | 50 |
| Llama 3 (Ollama) | 0.9 | 40 |
| Llama 3 (vLLM) | 1.0 | -1 (disabled) |
OpenAI does not expose top_k at all — nucleus sampling via top_p is the only truncation lever. Anthropic sets top_p just under 1.0 and a relatively high top_k of 250, effectively letting the model consider a broad vocabulary while still clipping the extreme tail. Llama 3 deployments vary: Meta’s reference uses top_p 0.9 and top_k 50, while vLLM disables top_k by default and leaves top_p at 1.0.
Penalty parameters
| Model | frequency_penalty | presence_penalty | repetition_penalty |
|---|---|---|---|
| GPT-4o | 0 (default) | 0 (default) | Not exposed |
| Claude 3 | Not exposed | Not exposed | Not exposed |
| Llama 3 (most hosts) | 0 | 0 | 1.0 (disabled) |
OpenAI is the only major API offering frequency and presence penalties natively. Anthropic omits them entirely, relying on top_k and the model’s training to suppress repetition. Llama 3 hosts typically expose repetition_penalty (the CTRL/Transformer-XL style multiplier) rather than OpenAI’s additive penalties. The semantics differ: OpenAI’s penalties add to logits, while repetition_penalty divides or multiplies probabilities for already-generated tokens.
How defaults behave in practice
GPT-4o at temperature 1.0, top_p 1.0
With no truncation and full entropy, GPT-4o produces diverse, sometimes surprising completions. For creative writing or brainstorming this works well. For structured extraction or code generation, you’ll want temperature 0.2–0.4 and top_p 0.9–0.95. The absence of top_k means you cannot clamp the vocabulary size directly — only via top_p.
Claude 3 at temperature 1.0, top_p 0.999, top_k 250
The near-1.0 top_p combined with top_k 250 creates a soft ceiling: the model considers the top 250 tokens by probability, then applies nucleus sampling within that set. This tends to produce coherent, on-track output with less “weirdness” than raw temperature 1.0 on other models. For deterministic tasks, drop temperature to 0.1–0.3 and keep top_k at 250 or lower it to 50.
Llama 3 at temperature 0.6–0.8, top_p 0.9, top_k 40–50
Meta’s recommended defaults (0.6 / 0.9 / 50) yield noticeably tighter, more focused completions than the other two at their defaults. The lower temperature and aggressive top_k truncation reduce hallucination rates on knowledge tasks but can make creative writing feel stiff. Many teams raise temperature to 0.8–1.0 for chat and drop top_k to 20–30 for code.
One comparison table
| Dimension | GPT-4o | Claude 3 | Llama 3 (managed) | Llama 3 (self-hosted) |
|---|---|---|---|---|
| Temperature default | 1.0 | 1.0 | 0.6–0.8 | 0.8–1.0 |
| Top-p default | 1.0 | 0.999 | 0.9 | 1.0 |
| Top-k default | N/A | 250 | 40–50 | -1 (disabled) |
| Frequency penalty | ✓ (additive) | ✗ | ✗ | ✗ |
| Presence penalty | ✓ (additive) | ✗ | ✗ | ✗ |
| Repetition penalty | ✗ | ✗ | ✗ | ✓ (multiplicative) |
| Min-p / typical-p | ✗ | ✗ | Sometimes | Often |
| Mirostat | ✗ | ✗ | Rare | Sometimes |
| Parameter stability | High (single provider) | High (single provider) | Medium (varies by host) | Low (varies by server) |
| Deterministic seed support | ✓ (seed param) | ✗ | Sometimes | Sometimes |
Latency and throughput implications
Sampling parameters affect latency indirectly. Lower top_k values reduce the softmax computation over the vocabulary — some inference engines (vLLM, TGI) skip logit calculation for tokens outside the top-k set. At top_k 50 vs. 250, you can see 5–15% decode speedup on 70B models. Top_p has no such optimization in most engines; it’s applied post-softmax.
Temperature does not change latency. Penalties add negligible overhead (a few microseconds per token for the logit adjustment). The real latency lever is max_tokens and whether you stream.
Ergonomics and ecosystem
OpenAI’s parameter set is the de facto reference — most SDKs, eval frameworks, and logging tools assume temperature, top_p, frequency_penalty, presence_penalty. If you build a prompt that relies on presence_penalty to suppress list repetition, porting to Claude or Llama 3 requires rewriting the prompt or adding a post-processing step.
Anthropic’s top_k is a powerful knob once you learn it. For RAG pipelines where you want the model to stick close to retrieved context, setting top_k 10–20 with temperature 0.1 often outperforms fiddling with top_p alone.
Llama 3’s fragmentation is the main ergonomic cost. A prompt tuned on Together’s defaults (temp 0.7, top_p 0.9, top_k 50) will behave differently on Fireworks (temp 1.0, top_p 1.0, top_k -1) or your own vLLM deployment. Pin the parameters explicitly in your client code; never rely on server defaults for production workloads.
Limits and guardrails
All three model families clamp temperature to [0, 2] (OpenAI) or [0, 1] (Anthropic). Top_p is [0, 1] everywhere. Top_k is typically [1, vocab_size] or -1 for disabled. OpenAI’s penalties accept [-2, 2]. Llama 3’s repetition_penalty is usually [0.1, 2.0] with 1.0 as neutral.
None of the APIs validate that your parameter combinations make sense. You can set temperature 0.0 with top_p 0.1 — the top_p becomes a no-op since the argmax is deterministic. You can set frequency_penalty 2.0 and presence_penalty 2.0 simultaneously and get degenerate output. Test your combinations.
Which to choose by use case
Structured extraction / JSON mode / function calling
Use GPT-4o with temperature 0.0–0.1, top_p 0.95, and seed set. The deterministic seed plus narrow sampling gives reproducible schema adherence. If you must use Llama 3, set temperature 0.1, top_p 0.9, top_k 20, and repetition_penalty 1.05.
Code generation
Use GPT-4o or Claude 3.5 Sonnet at temperature 0.2, top_p 0.95. For Llama 3, temperature 0.3, top_p 0.9, top_k 30. Avoid presence_penalty — it hurts syntax validity more than it helps.
Creative writing / brainstorming
Use GPT-4o at temperature 0.9–1.0, top_p 0.95, presence_penalty 0.3–0.5. The additive penalties nudge toward novelty without collapsing coherence. Claude 3 Opus at temperature 1.0, top_k 100 works well if you prefer its voice. Llama 3 70B at temperature 0.8, top_p 0.95, top_k 50 is a solid open-weight alternative.
RAG / grounded QA
Use Claude 3 at temperature 0.1, top_k 10–20. The low top_k forces the model to stay in the high-probability region conditioned on your context. GPT-4o at temperature 0.1, top_p 0.9 works but lacks the hard vocabulary clamp. Llama 3 at temperature 0.1, top_p 0.9, top_k 20, repetition_penalty 1.1.
High-throughput classification / routing
Use any model at temperature 0.0 with a fixed seed where supported. For Llama 3 self-hosted, add top_k 1 (greedy) and disable all penalties. The latency win from top_k 1 is measurable at scale.
Long-context summarization
Use GPT-4o at temperature 0.3, top_p 0.9, presence_penalty 0.2 to discourage looping. Claude 3 at temperature 0.3, top_k 50. Llama 3 at temperature 0.3, top_p 0.9, top_k 40, repetition_penalty 1.05.
The practical takeaway
Default sampling settings by model are not portable. A prompt that works at OpenAI’s defaults will drift on Claude and behave differently again on each Llama 3 host. Treat sampling parameters as part of your prompt contract — version them, test them, and pin them in your client configuration. The few lines of explicit parameter passing save hours of debugging when a provider updates their defaults or you migrate endpoints.