Llama 4 Scout's 10M context: which providers actually support it

Most teams evaluating Llama 4 Scout 10M context window providers discover the hard way that the advertised number and the accepted max_input_tokens are not the same. The model architecture supports ten million tokens, but the inference stack underneath it decides whether your request ever reaches that limit. This analysis cuts through the marketing and looks at what actually happens when you send a long prompt to the major endpoints.

Why 10M is a systems problem, not a weights problem

The Llama 4 Scout weights ship with a rotary position embedding scheme that scales to 10M positions. That does not mean any GPU cluster will allocate a KV cache for 10M tokens on demand. For a 17B-active MoE with 48 layers, the KV cache per token in bf16 runs into multiple megabytes. Multiply by 10M and you need terabytes of high-bandwidth memory just to hold attention state. No single commodity node does that today.

Providers that claim support either use multi-node pipeline parallelism with offloaded KV, approximate attention (sink tokens + windowed local attention), or they lie. The distinction matters because the fallback behavior changes your application’s correctness.

Three tiers of “support”

When a vendor lists Llama 4 Scout in their model catalog, read the fine print:

Tier 1: Native full-context

The endpoint accepts up to 10M tokens in one request and runs the full attention pattern (or a documented approximation that covers the whole sequence). Latency is brutal, but the model sees everything.

Tier 2: Context caching with capped live window

You preload a document into a cached session; the provider stores KV or embeddings. The interactive turn is limited to a sliding 128K–1M window. This is fine for “ask questions about this 500-page PDF” but fails if you need cross-document reasoning in a single prompt.

Tier 3: Silent truncation

The API accepts the request, drops everything past max_position_embeddings (often 131072), and returns a completion. No error. Your eval silently scores on the first chapter of War and Peace.

How to verify a provider’s real limit

Don’t trust the docs. Probe the endpoint. Most OpenAI-compatible servers expose a /v1/models route; some attach vendor-specific metadata.

curl -s https://api.my-provider.com/v1/models \
  -H "Authorization: Bearer $TOKEN" | jq '.data[] | select(.id | contains("llama-4-scout"))'

A honest response includes a field like:

{
  "id": "llama-4-scout-10m",
  "context_window": 10000000,
  "kv_offload": "distributed",
  "max_batch_tokens": 10000000
}

More commonly you get:

{
  "id": "llama-4-scout",
  "context_window": 131072
}

If the field is missing, send a canary: a 200K-token prompt of repeated unique strings and ask the model to repeat the 199,000th. If it hallucinates, you’re in Tier 3.

import openai, tiktoken

client = openai.OpenAI(base_url="https://api.my-provider.com/v1", api_key="...")
enc = tiktoken.get_encoding("cl100k_base")
# build 200k pseudo-unique tokens (truncate for test)
long_text = "TOKENXYZ " * 50000  # ~150k tokens, extend as needed
resp = client.chat.completions.create(
    model="llama-4-scout",
    messages=[{"role":"user","content": long_text + "\nWhat was the 49999th repeated word?"}]
)
print(resp.choices[0].message.content)

If the answer is wrong or the API returns context_length_exceeded, you know the ceiling.

The actual provider landscape

I won’t name every endpoint, but patterns are consistent across the Llama 4 Scout 10M context window providers I tested:

• First-party Meta deployment: The reference implementation supports the full window but is rate-limited and often gated to approved use cases. Throughput is low.
• Hyperscaler Bedrock/Vertex: They expose Scout with a 128K or 1M cap, citing “practical limits.” You get Tier 2 at best.
• Dedicated inference startups: Many list llama-4-scout with a 10M tag in marketing but the model card shows max_input_tokens: 131072. A few leverage vLLM with paged KV and will accept up to 1M with severe latency.
• OpenRouter-class gateways: These aggregate multiple backends. A gateway such as n4n.ai fronts 240+ models behind one OpenAI-compatible endpoint and will honor a routing directive like x-n4n-router: provider=foo while automatically falling back when that provider is degraded. That solves availability, not the underlying context cap—you still must pick a backend that allocates the KV.

The phrase “Llama 4 Scout 10M context window providers” is a spectrum. Only a minority run the full window natively.

Tradeoffs of shipping with 10M

Assume you find a Tier 1 endpoint. What then?

Latency: Prefill for 10M tokens is not seconds; it is minutes. A single user request ties up expensive accelerators. If you batch, you need sequence parallelism.

Cost: Per-token metering at 10M input tokens per call burns budget. Even at $0.01/1M input (a real low tier elsewhere), one call costs $0.10 just to read the prompt. Mistakes are expensive.

Quality: Long-context benchmarks show attention degradation even when the window is supported. The model may ignore middle sections (lost-in-the-middle). For retrieval, chunking + RAG beats a 10M dump.

# Better pattern for most apps: embed and retrieve
def answer_with_rag(query, doc_store, model="llama-4-scout"):
    chunks = doc_store.similarity_search(query, k=8)  # each ~2k tokens
    context = "\n".join(c.text for c in chunks)
    return client.chat.completions.create(
        model=model,
        messages=[{"role":"user","content": f"{context}\n\n{query}"}]
    )

This keeps live context under 32K and uses the model where it’s strong.

When 10M is the right call

There are cases: scanning an entire codebase for a specific anti-pattern, or legal discovery where you need cross-references across the full corpus in one reasoning pass. If you must, isolate those jobs to a Tier 1 provider, set a timeout of 300s+, and cache the KV for reuse.

{
  "model": "llama-4-scout-10m",
  "messages": [{"role":"user","content":"...10M tokens..."}],
  "cache_control": {"type": "ephemeral", "ttl": 3600}
}

Forwarding provider cache-control hints matters; some gateways pass that through, others drop it. Know which you use.

Decisive takeaway

Stop reading the hero numbers. The set of Llama 4 Scout 10M context window providers that genuinely accept ten million tokens in a single forward pass is small—first-party Meta, a couple of research-grade clusters, and perhaps one or two inference specialists with distributed KV. Everyone else truncates or windows. Verify with a canary request, design for Tier 2 by default, and reserve true 10M for batch jobs where latency and cost are acceptable. Your users won’t forgive silent truncation; your wallet won’t forgive careless 10M calls.