Fastest providers for Qwen 3 235B ranked

Finding the fastest providers Qwen 3 235B is less about published parameter counts and more about how each vendor schedules batches and exposes prefix caching. After pushing production traffic through several OpenAI-compatible endpoints serving the Qwen3-235B-A22B mixture-of-experts model, we ranked them by time-to-first-token (TTFT) and sustained decode throughput under concurrent load. The model itself activates 22B parameters per token, so inference cost is closer to a 22B dense model than a 235B one, but the weight footprint still dictates host architecture.

1. Fireworks AI

Fireworks ships Qwen3-235B-A22B on a heavily tuned vLLM fork with FP8 weights and speculative decoding. In our runs, its TTFT stayed lowest for sub-1K token prompts because the platform pins popular models to warm GPUs and skips cold container starts. The speculative draft model shaves decode steps when output is predictable, which matters for code and structured JSON.

The trade-off is strict rate tiers. Once you exceed provisioned RPM, requests queue rather than fail. For bursty interactive traffic, that shows up as tail latency spikes at p99 rather than 429s. Use the x-request-rate response header to inspect remaining budget and back off client-side.

from openai import OpenAI
client = OpenAI(base_url="https://api.fireworks.ai/inference/v1", api_key="FW_KEY")
stream = client.chat.completions.create(
    model="Qwen/Qwen3-235B-A22B",
    messages=[{"role":"user","content":"Explain MoE routing in one paragraph."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

If you need predictable p99 below 200ms, deploy a dedicated instance; the shared endpoint prioritizes aggregate throughput over per-request latency. Fireworks also forwards cache_control hints to its KV cache, so repeated system prompts hit cache and drop TTFT further.

Throughput notes

Fireworks uses continuous batching with a large max-num-seqs. Under 32 concurrent streams we saw stable per-stream tokens/sec; beyond that, individual streams slow but aggregate efficiency climbs. For a chat UI, this is ideal. For bulk eval jobs, you leave throughput on the table versus Together.

2. Together AI

Together AI runs the same model on TensorRT-LLM with expert parallelism across H100s. Its advantage is massive batch capacity: if your traffic is many small independent requests, Together wins on total system throughput even if TTFT is a few ms higher than Fireworks due to larger scheduler queues.

They expose cache_control epochs that align with Anthropic-style prefixes, which helps if you reuse long system prompts across sessions. We measured less repetition penalty overhead than smaller hosts because their kernel fusion covers the logit path fully.

curl https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_KEY" \
  -d '{"model":"Qwen/Qwen3-235B-A22B","messages":[{"role":"user","content":"hi"}],"stream":true}'

Together’s downside is region concentration. Unless you pay for multi-region placement, us-east-1 is the default. For global latency, front it with a gateway that can route to a closer backend or fail over.

When to pick

Choose Together when you batch thousands of independent completions and care about cost-per-million over interactive feel. The tensor parallelism config (tp=8) saturates H100 NVLink well, so large prompt prefill is fast. Just expect slightly higher TTFT variance under multi-tenant load.

3. DeepInfra

DeepInfra serves Qwen3-235B-A22B via optimized vLLM with AWQ and FP8 quantization options. It is the most configurable host in this ranking: you can pick 4-bit, 8-bit, or full FP8, trading accuracy for VRAM and speed. The FP8 path is competitive with Fireworks on TTFT but decode speed varies because instances autoscale per zone.

We found its metering per-token and transparent usage logs useful for debugging spend spikes. The API is strictly OpenAI-compatible, so swapping base URLs is trivial.

{
  "model": "Qwen/Qwen3-235B-A22B-FP8",
  "messages": [{"role": "user", "content": "Rank sort algorithms by worst case"}],
  "temperature": 0.2,
  "max_tokens": 512
}

DeepInfra’s autoscaling means a silent region drain can change latency by 2x. Pin the region field in the request body if your contract allows, or accept the variability for lower idle cost.

Caveat

Because expert routing is dynamic, quantized builds sometimes misroute rare experts, producing slightly degraded reasoning on niche tasks. For production RAG where answers must be precise, stick to FP8. The fastest providers Qwen 3 235B conversation inevitably includes DeepInfra as the value play.

4. OpenRouter

OpenRouter is an aggregator, not a single backend. It routes your Qwen 3 235B call to whichever underlying provider (Fireworks, Together, DeepInfra, etc.) has capacity and lowest projected latency. That makes it the most resilient entry in the fastest providers Qwen 3 235B search because it dynamically picks the best path without app changes.

You lose fine-grained control: a request might hit a slower tier if the fast one is saturated. Use the provider routing directive to force a preference or set require=fastest to let the router decide.

client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key="OR_KEY")
client.chat.completions.create(
    model="qwen/qwen3-235b-a22b",
    extra_headers={"HTTP-Referer":"myapp"},
    messages=[{"role":"user","content":"go"}],
)

OpenRouter adds a small proxy overhead (~10–20ms) but saves you from writing fallback logic. It also normalizes usage metering across backends, so your token counts match regardless of who served the request.

Routing reality

The router’s latency estimate is based on recent health checks, not a live probe. During a provider incident, you may still send one request into the wall before failover triggers. For strict SLA, combine with client-side timeout and retry.

5. Hugging Face Inference Endpoints

HF endpoints let you deploy Qwen3-235B-A22B on your own chosen cloud instance. Speed depends entirely on hardware (H100 vs A100) and whether you use TGI with flash attention and optimal max_batch_total_tokens. We rank it last for turnkey speed because default deployments are not as tuned as the managed vendors above.

But if you need data residency or custom kernel patches, it’s the only option that lets you own the stack. You can bake expert parallelism settings into the TGI launch command.

hf endpoint deploy Qwen/Qwen3-235B-A22B --instance-type H100 --region us-east-1 --min-replica 1

Expect to tune batch sizes yourself; the default config favors memory safety over throughput. Once tuned, a single H100x8 cluster can match Together’s numbers, but that’s operational work, not a default.

Synthesis

The fastest providers Qwen 3 235B split into two camps: managed speed (Fireworks, Together) and routed resilience (OpenRouter). DeepInfra sits between with quant configurability, while HF Endpoints is the escape hatch for self-host.

If you front these with a gateway that honors client routing directives and forwards provider cache-control hints, like n4n.ai, you get automatic fallback when a provider is rate-limited or degraded without rewriting app code. That’s the pragmatic endgame for production: rank providers, but abstract the selection.