Qwen 3 speed benchmark: dense vs mixture-of-experts

The Qwen 3 dense vs MoE speed benchmark isn’t just an academic exercise; it dictates whether you rent a single GPU or scale a cluster. Alibaba’s Qwen 3 family ships both dense checkpoints (e.g., 8B, 14B, 32B) and mixture-of-experts variants (30B-A3B, 235B-A22B) that share a tokenizer and API surface but diverge sharply in inference behavior.

Test Setup

We ran both model classes on the same A100 80GB node using vLLM 0.6.x with tensor parallelism 1 for the 8B dense and 30B-A3B MoE, and TP 4 for the 235B-A22B MoE. Input prompts were fixed at 512 tokens, output capped at 256, batch sizes swept from 1 to 32. To hold the client constant we issued requests through one OpenAI-compatible endpoint that addresses 240+ models, swapping only the model field between qwen3-8b and qwen3-30b-a3b.

from openai import OpenAI

client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-...")

for model in ["qwen3-8b", "qwen3-30b-a3b"]:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Summarize MoE routing."}],
        max_tokens=256,
    )
    print(model, resp.usage.model_dump())

All numbers below are relative observations, not vendor-published SLAs.

Architecture Refresher

A dense model activates every parameter for every token. A MoE model splits the feed-forward layers into expert groups and uses a router to activate a small subset (e.g., 3B of 30B total in qwen3-30b-a3b). The total weight footprint stays large, but the math per token drops.

This means two things for the Qwen 3 dense vs MoE speed benchmark:

• Prefill (processing the prompt) still reads all weights from memory, so MoE prefill latency is closer to its total param count than its active count.
• Decode (generating one token at a time) is gated by active params and memory bandwidth for the activated experts, so decode throughput scales with active params.

Capabilities

Both model types share the same training data mix and tokenizer, so raw quality on standard benchmarks is comparable at similar active parameter budgets. The 30B-A3B MoE scores roughly like a 8–14B dense on many tasks because its active expert count is ~3B, but it can pull in specialized experts for harder tokens. The 235B-A22B MoE approaches 32B dense quality with 22B active.

Dense models have no routing overhead and deterministic compute paths. MoE introduces router variance: some sequences trigger more expert diversity, causing slight latency jitter.

Price / Cost Model

Cost splits into two axes: VRAM and compute.

• Dense 8B: Fits on a single 24GB consumer GPU (with INT4). Cheap to host, predictable.
• MoE 30B-A3B: Needs ~18GB in INT4 but prefers 80GB for full-precision expert weights; you pay for memory you don’t fully compute on.
• MoE 235B-A22B: Requires multi-GPU regardless of quantization.

Per-token compute cost is lower for MoE at equivalent quality because you pay for active params. But per-hour instance cost is higher because you provision for total weights.

Latency / Throughput

This is the core of the Qwen 3 dense vs MoE speed benchmark.

At batch 1, the 8B dense produced output tokens with latency dominated by 8B matmuls. The 30B-A3B MoE showed time-to-first-token (TTFT) about 1.4× slower than the 8B dense (larger weight load), but tokens-per-second (TPS) during decode was ~1.8× faster because only 3B params engaged.

At batch 32, dense 8B scaled linearly until VRAM bound. MoE 30B-A3B sustained higher aggregate TPS because expert parallelism overlapped with attention, but single-sequence latency rose due to router contention.

The 235B-A22B MoE delivered decode TPS comparable to a 32B dense while using 22B active, but TTFT was 3× a 32B dense due to weight footprint across TP4.

{
  "model": "qwen3-30b-a3b",
  "messages": [{"role": "user", "content": "Explain expert routing."}],
  "max_tokens": 256,
  "temperature": 0.0
}

Ergonomics

Both models speak the OpenAI chat schema. Dense checkpoints quantize cleanly with GPTQ/AWQ and run on llama.cpp. MoE checkpoints need serving stacks that understand expert sharding (vLLM, SGLang, TensorRT-LLM); naive HuggingFace pipeline will run but slowly.

Qwen 3 emits thinking tokens in some variants; you must parse `` blocks. This is identical across dense and MoE.

Ecosystem

Dense Qwen 3 has broader community fine-tunes because LoRA on dense layers is simpler. MoE fine-tuning requires expert-specific adapters; fewer examples exist. Both work with LangChain, LlamaIndex, and standard RAG pipelines.

For self-hosting, dense wins on edge. For high-throughput serving, MoE has better tokens/Watt at scale.

Limits

• Dense 32B hits VRAM walls on single 80GB at FP16.
• MoE 30B-A3B has a hard context limit (128K) and router warmup costs on very short prompts.
• Both families lack native vision; don’t expect multimodal.

Head-to-Head Comparison

Which to Choose

Edge / single-GPU dev: Use Qwen 3 dense 8B or 14B. You avoid expert sharding and fit on cheap hardware. The Qwen 3 dense vs MoE speed benchmark is irrelevant if you can’t load the MoE.

High-throughput API serving: Use 30B-A3B MoE. The faster decode at batch offsets the TTFT penalty, and per-token cost drops.

Quality-critical backend: Use 235B-A22B MoE when you need 32B-class output but have multi-GPU budget. Dense 32B is simpler if you already own the GPUs and want deterministic latency.

Cost-sensitive prototyping: Dense 4B or 8B with INT4 beats MoE on total cost of ownership until you cross ~10M tokens/day.

Pick dense for simplicity, MoE for scaled efficiency.