Doing a pay-per-token pricing comparison across LLM providers is the fastest way to find where your inference budget leaks. Headline rates per million tokens obscure caching discounts, context-window penalties, and tokenizer efficiency, all of which shift the real cost per request.
Providers and Models Under Test
We compared four providers that ship production-grade closed and open-weight models with transparent per-token billing:
- OpenAI – GPT-4o and GPT-4o mini
- Anthropic – Claude 3.5 Sonnet
- Google – Gemini 1.5 Pro
- Mistral AI – Mistral Large (via La Plateforme)
All four publish input/output token prices. None charge for provisioning; you pay only for what you send and receive.
Head-to-Head Dimensions
Capabilities
OpenAI GPT-4o is multimodal (text, vision, audio) and has the most mature function-calling ecosystem. Claude 3.5 Sonnet leads on instruction following and long-form reasoning, with a 200K context window. Gemini 1.5 Pro stretches to 1M+ tokens and handles interleaved text and images natively. Mistral Large is competitive on multilingual text and offers open-weight variants for self-hosting.
Price/Cost Model
Published list rates (mid-2024) for the flagship models, per 1M tokens:
- OpenAI GPT-4o: $2.50 in / $10 out
- Anthropic Claude 3.5 Sonnet: $3 in / $15 out
- Google Gemini 1.5 Pro (≤128K context): $1.25 in / $5 out
- Mistral Large: $2 in / $6 out
A pay-per-token pricing comparison must also factor prompt caching. Anthropic gives 90% discount on cached input tokens; OpenAI offers similar via its cached tokens beta; Gemini charges $1.25/1M for cached context after first write. Batch inference cuts cost 50% on all three US providers but adds latency.
Latency/Throughput
Measured p50 latency varies by region and load. OpenAI and Anthropic return first token in roughly 300–600ms for small prompts on paid tiers. Gemini’s large-context calls incur higher prefill time. Mistral’s hosted API is competitive but throughput depends on tier. None of these are hard guarantees; all expose rate limits that throttle bursty traffic.
Ergonomics
OpenAI’s SDK is the de facto standard; most frameworks target it first. Anthropic’s SDK is clean but diverges on message shapes. Google’s generative-ai client is verbose. Mistral’s API is OpenAI-compatible, easing migration.
A gateway such as n4n.ai collapses these into one OpenAI-compatible endpoint with automatic fallback when a provider is rate-limited, which removes per-vendor retry logic from your client.
Ecosystem
OpenAI: largest plugin/tooling mesh, fine-tuning, embeddings. Anthropic: strong safety tooling, prompt caching, fine-tuning beta. Google: Vertex integration, TPU-optimized serving. Mistral: open weights, on-prem flexibility, EU data residency.
Limits
Context windows: GPT-4o 128K, Claude 3.5 200K, Gemini 1.5 Pro 1M+, Mistral Large 32K (La Plateforme). Max output tokens: 4K–8K typical. Rate limits: OpenAI tier-based (e.g., 10K TPM initial), Anthropic 40K TPM on pro, Gemini 60 req/min, Mistral varies by plan.
Comparison Table
| Provider | Capabilities | Price (in/out per 1M) | Latency | Ergonomics | Ecosystem | Context Limit |
|---|---|---|---|---|---|---|
| OpenAI GPT-4o | Multimodal, best tooling | $2.50 / $10 | Low p50 | OpenAI SDK std | Huge, fine-tune | 128K |
| Anthropic Claude 3.5 | Reasoning, 200K | $3 / $15 | Low–med | Custom SDK | Safety, cache | 200K |
| Google Gemini 1.5 Pro | 1M+ ctx, multimodal | $1.25 / $5 | Med (prefill) | Verbose SDK | Vertex, TPU | 1M+ |
| Mistral Large | Multilingual, open wt | $2 / $6 | Med | OpenAI-compat | EU, self-host | 32K |
Estimating Real Cost in Code
Token counts differ across tokenizers. Use provider-native counters where possible.
# Approximate cost calculator
PRICING = {
"gpt-4o": {"in": 2.50, "out": 10.0},
"claude-3.5-sonnet": {"in": 3.0, "out": 15.0},
"gemini-1.5-pro": {"in": 1.25, "out": 5.0},
"mistral-large": {"in": 2.0, "out": 6.0},
}
def cost_usd(model, in_tokens, out_tokens):
p = PRICING[model]
return (in_tokens / 1e6) * p["in"] + (out_tokens / 1e6) * p["out"]
# Example: 10K in, 2K out on Gemini
print(cost_usd("gemini-1.5-pro", 10_000, 2_000)) # ~0.0225For OpenAI, count tokens precisely:
import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
n = len(enc.encode("System prompt plus user text"))Anthropic does not publish its tokenizer; estimate with ~4 chars/token. Google provides gemini.token_count in its SDK.
Optimizing Beyond the Headline Rate
A pay-per-token pricing comparison is incomplete without caching and batching. If you send the same system prompt on every call, cache it:
{
"anthropic-beta": "prompt-caching-2024-07-31",
"system": [
{"type": "text", "text": "You are a strict JSON parser.", "cache_control": {"type": "ephemeral"}}
]
}On OpenAI, set cached_tokens via the responses API beta. Gemini uses context_cache objects billed at write time.
Routing smaller tasks to mini models (GPT-4o mini at $0.15/$0.60) or Mistral 8x22B (cheaper open) cuts spend 10x. Gateways that honor client routing directives let you pin a model per request without code changes.
Which to Choose
High-volume, cost-sensitive text: GPT-4o mini or Mistral Large via OpenAI-compatible endpoint. Use batch API for async jobs.
Long-context analysis (legal, repo scans): Gemini 1.5 Pro for up to 1M tokens at lowest rate, or Claude 3.5 Sonnet if you need stronger reasoning over the retrieved span.
Highest quality agentic loops: Claude 3.5 Sonnet or GPT-4o. Pay the output premium only where token savings from fewer retries justify it.
EU residency / self-host: Mistral Large or self-hosted Mistral weights; avoid cross-region egress fees.
Mixed traffic with SLA needs: Put a fallback gateway in front. If one provider 429s, the request shifts without client changes, and per-token metering stays consistent.
The raw pay-per-token pricing comparison shows Gemini cheapest on paper, but tokenizer and cache behavior decide the bill. Profile your real prompts before committing.