The choice between Meta’s two Llama 4 mixture-of-experts releases is rarely about raw parameter counts. For anyone shipping inference, the operative question is Llama 4 Maverick vs Scout speed: how many tokens per second you get, what latency you see at the first byte, and how those numbers hold up under concurrency. Both models activate 17B parameters per token, but their total footprints and context windows create distinct performance profiles.
Architecture and active compute
Llama 4 Maverick packs 400B total parameters across 128 experts, with 17B active per token and a 1M token context window. Llama 4 Scout is smaller at 109B total across 16 experts, same 17B active per token, but extends to 10M context. The shared active parameter count means the matmul workload for a single token is nearly identical on paper.
That similarity is why Llama 4 Maverick vs Scout speed is not a simple “big model is slower” story. Decode step latency is dominated by moving the active weights from HBM to compute, not by the silent experts sitting idle. The difference is that Maverick’s 400B weight set requires roughly 4x the VRAM to keep resident, which constrains how many replicas you can pack per GPU node. Scout’s 16 experts are easier to shard across fewer devices without pipeline bubbles.
MoE routing adds a small per-token dispatch cost. With 128 experts, Maverick’s router must score and select across a wider pool, but the top-k activation (typically 2) keeps compute constant. In practice the routing overhead is microseconds, not milliseconds, yet under heavy batching it aggregates.
Cost model and token economics
API providers price these models per input and output token. Because the active compute is equal, the marginal cost of generating a token is close, but Maverick’s larger memory footprint pushes hosting cost up, and that surfaces as a higher per-token rate on most endpoints. Scout’s longer context also introduces KV cache cost: storing attention state for millions of tokens is not free, and providers either charge hidden cache fees or cap usable context.
If you self-host, Maverick needs more GPUs just to load, while Scout fits on fewer. That capital difference dwarfs the tiny per-token compute gap. Per-token usage metering on a gateway lets you attribute spend precisely:
{
"usage": {
"prompt_tokens": 128,
"completion_tokens": 256,
"total_tokens": 384
}
}The total_tokens field is identical in structure for both models; only the billed rate differs upstream.
Latency, throughput, and concurrency
Under a single concurrent stream, both models deliver similar time-to-first-token (TTFT) for short prompts because prefill cost scales with active params × sequence length. Where they diverge is under batching. Maverick’s larger expert count (128 vs 16) adds a small routing overhead, but the real penalty is memory bandwidth contention when many requests hit different experts—the gateway must fetch from a wider weight pool. Scout’s 16 experts are easier to keep hot.
Long-context workloads flip the script. Scout’s 10M window means KV cache grows linearly; at high batch sizes the cache steals HBM from weights, forcing evictions and raising decode latency. Maverick’s 1M cap is easier to manage.
Here is a minimal async probe to measure decode throughput against an OpenAI-compatible endpoint:
import asyncio, time, openai
async def bench(model, prompt, tokens=256):
client = openai.AsyncOpenAI(base_url="https://api.example.com/v1", api_key="KEY")
start = time.monotonic()
stream = await client.chat.completions.create(
model=model, messages=[{"role":"user","content":prompt}],
max_tokens=tokens, stream=True
)
n = 0
async for chunk in stream:
if chunk.choices[0].delta.content:
n += 1
dt = time.monotonic() - start
return n / dt
for m in ["llama-4-maverick", "llama-4-scout"]:
print(m, asyncio.run(bench(m, "Explain MoE routing.")))When you run this across providers, an OpenAI-compatible gateway such as n4n.ai that honors client routing directives and forwards provider cache-control hints lets you isolate model behavior from provider-specific batching tricks.
For concurrency testing, launch parallel coroutines:
async def load_test(model, concurrency=32):
tasks = [bench(model, "Summarize this.") for _ in range(concurrency)]
return await asyncio.gather(*tasks)Scout typically sustains higher concurrency on the same hardware because its narrower expert set reduces weight-fetch scatter.
Ergonomics and API surface
Both models speak the standard chat completion schema. Tool calling works on Maverick; Scout supports it but with less rigorous function adherence in practice. JSON mode is available on both via response_format. Scout’s long context is ergonomic for RAG: you can dump entire codebases in one prompt without chunking.
{
"model": "llama-4-scout",
"messages": [{"role":"user","content":"Summarize the repo"}],
"response_format": {"type":"json_object"},
"max_tokens": 1024
}Maverick’s multimodal input accepts images, which Scout also supports but with weaker visual grounding. Streaming, stop sequences, and logprobs behave identically.
Ecosystem and deployment limits
Maverick is the default high-quality option on most hosted platforms; Scout is marketed for long-context verticals. Provider availability varies: some gateways expose Scout only with truncated 1M context despite the 10M claim. Maverick’s 1M context is widely honored.
Fine-tuning recipes for Scout are emerging because its smaller total size is cheaper to adapt. Maverick fine-tunes are rarer due to GPU requirements. Both are served behind OpenAI-compatible endpoints on aggregators, which simplifies failover.
Head-to-head comparison
| Dimension | Llama 4 Maverick | Llama 4 Scout |
|---|---|---|
| Capabilities | 400B MoE, 128 experts, multimodal, 1M ctx, stronger reasoning | 109B MoE, 16 experts, multimodal, 10M ctx, long-doc focus |
| Price/cost model | Higher per-token rate due to VRAM footprint; similar compute cost | Lower hosting cost; KV cache cost at extreme context |
| Latency/throughput | Near-equal single-stream decode; wider expert pool hurts high-batch | Equal decode; 16 experts batch-friendly; long ctx KV pressure |
| Ergonomics | Tool calling solid, JSON mode, image input | Tool calling weaker, 10M ctx simplifies RAG, image input |
| Ecosystem | Broad provider support, 1M ctx honored | Patchy 10M support, cheaper to fine-tune |
| Limits | Needs many GPUs to load; 1M ctx max | Usable context often capped by provider; 16 experts |
Which to choose
Low-latency interactive chat: Either works. If you need best reasoning per token and can afford the replica count, Maverick. If you want to pack more streams per node, Scout.
Long-document RAG without chunking: Scout is the only native option with 10M context, but verify your provider actually serves it. Otherwise Maverick at 1M is safer.
Multimodal product: Maverick has the more reliable vision grounding today. Scout can take images but expect rougher outputs.
Cost-sensitive batch jobs: Scout’s smaller total footprint reduces infrastructure spend; per-token speed is a wash.
High-concurrency serving: Scout’s 16-expert design is easier to keep warm, giving more predictable p99 latency under load.
The Llama 4 Maverick vs Scout speed debate is less about raw velocity and more about fitting the model to your memory and context envelope. Pick based on context length and batch density, not on the assumption that bigger means slower.