DeepSeek R1 distilled models: API access compared

DeepSeek R1 distilled models API access is fragmented: the open-weight checkpoints are served by at least four distinct providers, each with its own endpoint shape, rate limits, and cost structure. This head-to-head strips away the marketing so you can wire up the right one without guessing.

The contenders

DeepSeek released the R1 distilled weights (Llama-70B, Llama-8B, Qwen-32B, and smaller Qwen variants) but does not host them on its own paid API—the official api.deepseek.com only serves the full R1 and V3 chat models. To call a distill over HTTP you pick between:

• OpenRouter – aggregator that resells many hosted copies behind one OpenAI-compatible API.
• Together AI – GPU cloud that hosts the weights natively with its own and OpenAI-compatible endpoints.
• Fireworks AI – inference optimizer with FireAttention kernels and an OpenAI-compatible route.
• Local Ollama – self-hosted HTTP server wrapping the GGUF or safetensors weights.

A gateway like n4n.ai collapses the first three into a single OpenAI-compatible endpoint with automatic fallback when a backend is degraded, but the underlying tradeoffs remain the same.

Capabilities and model variants

All providers expose the 70B Llama distill and the 32B Qwen distill; the 8B and 14B variants are spottier. Context windows follow the base model: Llama distills advertise 128K, Qwen distills 32K–128K depending on the serving config. In practice, OpenRouter and Fireworks cap requests at 32K tokens unless you pass an explicit max_tokens under the limit.

The distills are instruction-tuned chat models, not raw reasoning engines. They accept standard messages arrays and emit a single completion. None support native tool calling as reliably as the full R1; if you need function calling, wrap them with a client-side parser.

{
  "model": "deepseek-r1-distill-llama-70b",
  "messages": [{"role": "user", "content": "Explain beam search."}],
  "max_tokens": 1024,
  "temperature": 0.3
}

Pricing and cost model

Every hosted option charges per token with no minimums. OpenRouter and Fireworks publish floating rates that track upstream GPU cost; Together uses fixed tiers. The distills run roughly 5–10x cheaper than the 671B full R1 per million tokens because they fit on a single 8-GPU node.

Local Ollama has zero per-token cost but requires you to own the hardware. A 70B distill in 4-bit needs ~40 GB VRAM; that’s a single A100 80GB or two 3090s. If you already have the box, the marginal cost is electricity.

None of the hosted providers charge for cached prompts on these models yet. n4n.ai forwards provider cache-control hints but the distill endpoints themselves don’t implement prompt caching, so you pay full price on every repeat.

Latency and throughput

Fireworks wins single-stream latency: its custom CUDA graphs cut time-to-first-token to sub-200ms on the 70B on A100. Together prioritizes batch throughput; a single request may wait 300–500ms behind a batch but sustains higher aggregate tokens/sec. OpenRouter latency depends on which backend it routes to—sometimes Fireworks, sometimes a smaller provider—so it varies request to request.

Local latency is bounded by your GPU. On a single 3090, the 70B 4-bit distill yields ~12 tokens/sec decode; fine for interactive use, terrible for bulk.

# Fireworks OpenAI-compatible call
curl https://api.fireworks.ai/inference/v1/chat/completions \
  -H "Authorization: Bearer $FW_KEY" \
  -d '{"model":"accounts/fireworks/models/deepseek-r1-distill-llama-70b","messages":[{"role":"user","content":"hi"}]}'

Ergonomics and API design

All three cloud providers mirror the OpenAI chat completions schema. You swap base_url and model string; the rest of your SDK code stays.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.together.xyz/v1",
    api_key="TOGETHER_KEY",
)
r = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-Distill-Llama-70b",
    messages=[{"role": "user", "content": "Summarize RFC 7231"}],
)
print(r.choices[0].message.content)

Ollama uses a similar but non-identical REST shape:

curl http://localhost:11434/api/chat -d '{
  "model": "deepseek-r1-distill-llama-70b",
  "messages": [{"role":"user","content":"ping"}],
  "stream": false
}'

Streaming works everywhere. OpenRouter and n4n.ai honor stream: true and SSE; Together and Fireworks do too. If you need provider-specific routing (e.g., “never use provider X”), OpenRouter supports a provider object; n4n.ai honors client routing directives in the same field.

Ecosystem and tooling

LangChain and LlamaIndex have off-the-shelf ChatOpenAI wrappers that work against any of the cloud endpoints by setting openai_api_base. Ollama has first-class Python and JS libraries. Together ships its own together SDK with async support; Fireworks documents a thin openai client extension.

Model cardinality is the differentiator: OpenRouter lists four distill SKUs; Together lists two; Fireworks lists one or two. If you want to A/B the Llama vs Qwen distill without code changes, OpenRouter or a gateway is simpler.

Limits and rate limits

OpenRouter enforces a credit-based limit; new accounts get $5 and a low RPM (requests per minute) that rises as you spend. Together starts at 20 RPM on free tier, 600+ on paid. Fireworks defaults to 100 RPM but will raise on request. Ollama has no rate limit—your thermal envelope is the limit.

All cloud providers cap max_tokens per request (usually 4K–32K). None allow parallel function calls on these distills.

Side-by-side comparison

Which to choose

Prototyping a single-feature app: Use Fireworks. Lowest latency, minimal config, OpenAI-compatible. You’ll ship in an hour.

Multi-model experimentation: OpenRouter or a gateway such as n4n.ai. One key, four model strings, easy swap between Llama and Qwen distills without touching base URLs.

High-volume batch jobs: Together. Its throughput orientation and fixed pricing make cost predictable when you’re processing millions of rows overnight.

Privacy or air-gapped deployment: Ollama. Run the 32B Qwen distill on a single 24GB card; no data leaves the box. Accept the slower decode.

Production with fallback needs: A gateway that aggregates the above removes the single-provider outage risk. You get per-token metering and automatic reroute when one provider is rate-limited, at the cost of one extra hop.

Pick by deployment constraint first, then by latency. The model weights are identical; the API access layer is where the real differences live.