Cheapest tokens per second: provider price rankings

Engineering teams optimizing LLM spend need more than a static price sheet. The cheapest tokens per second provider rankings balance per-token cost against sustained generation speed, because a cheap model that crawls under load costs you latency and user experience. Below we rank eight providers by public pricing and observed throughput characteristics, so you can pick a supplier that fits both budget and responsiveness.

1. Groq (Llama 3 70B)

Groq’s LPU infrastructure is purpose-built for transformer inference, and it shows. Public rates for Llama 3 70B on Groq sit around $0.59 per 1M input tokens and $0.79 per 1M output tokens, with the 8B variant often free in preview tiers. The headline is speed: the same 70B model consistently streams hundreds of tokens per second, dwarfing typical A100/H100 GPU endpoints.

That combination puts Groq at the top of the cheapest tokens per second provider rankings for open-weight models. The tradeoff is model selection—you’re limited to what Groq has ported to its LPU pipeline, and enterprise controls are thinner than hyperscalers. For high-volume, latency-sensitive open-model workloads, it’s hard to beat.

# Effective cost per 1K generated tokens at observed throughput
price_per_1m_out = 0.79
tps = 300  # documented Groq Llama 3 70B range
cost_per_1k_tokens = price_per_1m_out / 1000
print(f"${cost_per_1k_tokens:.4f} per 1K tokens at {tps} tps")

2. Google Gemini 1.5 Flash

Gemini 1.5 Flash is Google’s price-performance entry: $0.075 per 1M input and $0.30 per 1M output tokens as listed in Google’s public docs. Throughput via the Vertex or AI Studio APIs is strong, with multi-second context loads but steady generation once prefill completes.

For many English and multilingual tasks, Flash delivers near-frontier quality at a fraction of the cost. It ranks high in the cheapest tokens per second provider rankings because the per-token rate is among the lowest of any managed proprietary model, and the infrastructure rarely throttles small accounts. Watch context caching billing if you reuse long prompts.

3. Anthropic Claude 3 Haiku

Claude 3 Haiku is Anthropic’s small model: $0.25 per 1M input, $1.25 per 1M output. That’s pricier per token than Gemini Flash, but Haiku’s generation speed is consistently high, and the model handles structured extraction better than many open equivalents.

If your workload needs reliable tool use with low latency, Haiku climbs the cheapest tokens per second provider rankings despite a higher nominal rate. The math works when you factor in fewer retries and cleaner parses.

4. OpenAI GPT-4o mini

GPT-4o mini is OpenAI’s budget option: $0.15 per 1M input, $0.60 per 1M output. Throughput on OpenAI’s shared infrastructure is moderate—usually tens of tokens per second, not hundreds—but the API is ubiquitous and stable.

For teams already on OpenAI, mini is a sensible default that appears in the middle of the cheapest tokens per second provider rankings. You trade raw speed for ecosystem maturity and fine-tuning support.

5. Fireworks AI (Llama 3 70B)

Fireworks lists Llama 3 70B at $0.90 per 1M input and $0.90 per 1M output, with Fireworks-specific optimizations that push throughput above generic vLLM deployments. Their distributed inference gives predictable latency for batch and online serving.

In the cheapest tokens per second provider rankings, Fireworks sits just behind Groq on speed but offers broader model coverage including embedding and vision. You pay a small premium over raw open-model cost for managed reliability.

6. Together AI (Mixtral 8x7B)

Together AI serves Mixtral 8x7B at roughly $0.60 per 1M input and $0.60 per 1M output. The MoE model runs efficiently on GPU clusters, delivering solid tokens-per-second for a 47B-active-param model.

Together’s appeal in the cheapest tokens per second provider rankings is the mix of cheap open weights and a large model zoo. Speed is decent but not LPU-class; expect 50–100 tps depending on batch.

7. Mistral AI (Mistral Small)

Mistral’s own hosted Small model is $2 per 1M input, $6 per 1M output—clearly not the cheapest per token. However, its EU hosting and strong coding make it relevant for compliance-bound teams.

It lands lower in the cheapest tokens per second provider rankings, but if data residency is a hard requirement, the effective cost of alternatives may be higher once legal overhead is counted.

8. DeepInfra (Qwen 2 72B)

DeepInfra offers Qwen 2 72B at about $0.90 per 1M input and $0.90 per 1M output. The service is barebones but globally distributed, with competitive generation rates for a 70B-class model.

For non-English workloads, Qwen often outperforms Llama at similar price, earning a spot in the cheapest tokens per second provider rankings for multilingual apps.

Synthesis

No single provider wins every axis. The table below summarizes public list prices and relative speed tier.

When you build against these numbers, treat them as a starting point. Provider outages and rate limits shift real-world cost. An OpenRouter-class gateway such as n4n.ai can honor your routing directives and automatically fall back when a chosen provider is degraded, keeping your cost-optimized pipeline running without manual intervention.

To compute your own effective ranking, use a simple metric: (1 / cost_per_token) * tps. Higher means more tokens per dollar-second.

def tokens_per_dollar_second(price_per_1m_out, tps):
    cost_per_token = price_per_1m_out / 1_000_000
    return (1 / cost_per_token) * tps

print(tokens_per_dollar_second(0.79, 300))  # Groq example

That’s the practical cut of the cheapest tokens per second provider rankings—pick by workload, not by headline price alone.