Mixtral 8x7B pricing: Together AI vs DeepInfra vs Groq

A Mixtral 8x7B API pricing comparison looks simple on paper—three hosts, one open-weight model—but the token rates, rate limits, and latency profiles diverge enough to change your architecture. This breakdown puts Together AI, DeepInfra, and Groq side by side on the axes that matter when you ship to production.

Capabilities and model variants

All three serve the same base weights: Mistral AI’s Mixtral 8x7B mixture-of-experts instruct model. The differences are in packaging.

Together AI exposes mistralai/Mixtral-8x7B-Instruct-v0.1 with support for fine-tuning and dedicated endpoints. DeepInfra serves the same checkpoint as a serverless inference endpoint. Groq compiles the model for its LPU accelerator and ships mixtral-8x7b-32768 with a 32k context window.

None of them alter the model’s license or core behavior. What changes is how you provision it:

# Together AI
from openai import OpenAI
together = OpenAI(
    base_url="https://api.together.xyz/v1",
    api_key="TOGETHER_KEY",
)
together.chat.completions.create(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    messages=[{"role": "user", "content": "Summarize this"}],
)

# DeepInfra
deepinfra = OpenAI(
    base_url="https://api.deepinfra.com/v1/openai",
    api_key="DI_KEY",
)
deepinfra.chat.completions.create(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    messages=[{"role": "user", "content": "Summarize this"}],
)

# Groq
groq = OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key="GROQ_KEY",
)
groq.chat.completions.create(
    model="mixtral-8x7b-32768",
    messages=[{"role": "user", "content": "Summarize this"}],
)

Price and cost model

Published rates as of early 2024 show a clear spread. Together AI listed Mixtral 8x7B at $0.60 per million input and output tokens. DeepInfra listed $0.27 per million for both. Groq matched DeepInfra at $0.27 per million and added a free tier with strict rate limits for non-commercial or low-volume use.

The Mixtral 8x7B API pricing comparison gets nuanced because all three charge per token, but only Groq bundles a usable free quota. If you process billions of tokens monthly, DeepInfra and Groq are roughly 55% cheaper than Together. If you need fine-tuning, Together’s premium may be justified.

Token accounting is identical across hosts—prompt tokens plus completion tokens. A gateway that fronts these with per-token usage metering lets you aggregate cost without writing three billing integrations.

Latency and throughput

Groq is the outlier. Its LPU delivers time-to-first-token often under 100 ms and sustained generation well above 500 tokens/s for this model. Together and DeepInfra run on GPU clusters (A100/H100 class) where TTFT lands in the 100–400 ms range and throughput varies with batch size and queue depth.

For interactive chat, Groq feels instant. For asynchronous summarization jobs, the GPU hosts are fine. The MoE architecture means only two experts (≈14B params) activate per token, so all three hit respectable speeds; Groq just removes the memory-bandwidth bottleneck.

Ergonomics

All three implement the OpenAI chat completions schema. You can swap base_url and model with zero changes to request shaping, streaming, or function-call plumbing.

curl https://api.groq.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $GROQ_KEY" \
  -d '{"model":"mixtral-8x7b-32768","messages":[{"role":"user","content":"ping"}]}'

Together and DeepInfra add provider-specific fields (e.g., repetition_penalty on DeepInfra, logprobs extensions on Together) but the common path is stable. Groq’s 32k context is the largest default among the three; the others default to 4k or 8k unless you pass max_tokens and a supported context extension.

Ecosystem and tooling

Together AI builds the deepest platform: fine-tuning UI, batch inference, and model marketplace. DeepInfra stays minimal—serverless endpoints, one YAML for custom models, usage dashboard. Groq focuses on raw inference speed and a small model catalog optimized for its silicon.

If your stack already uses LangChain or LlamaIndex, all three drop in as OpenAI-compatible backends. For observability, you’ll still need to tag requests per provider; none share a unified trace ID.

Limits and quotas

Together enforces tier-based requests-per-minute that scale with spend. DeepInfra caps concurrent serverless workers unless you rent dedicated instances. Groq’s free tier limits you to ~30 requests/minute and modest daily token caps; paid tier lifts these but still trails GPU hosts on total available concurrency per dollar.

Context length is a hard limit: exceed it and you get a 400. Groq’s 32k is forgiving; the others need explicit configuration for long prompts.

Head-to-head table

Which to choose

Low-latency user-facing chat: Groq. The LPU latency is unmatched and the free tier covers prototyping. Ship with a fallback to DeepInfra when Groq rate-limits.

Cost-sensitive batch pipelines: DeepInfra. At $0.27/M it matches Groq’s paid rate without LPU lock-in, and serverless means zero idle cost.

Custom fine-tunes or managed training: Together AI. You pay more per token but get in-place fine-tuning on the same endpoint you infer from.

Multi-provider production: Front all three with a routing layer. A gateway such as n4n.ai honors client routing directives and provides automatic fallback when a provider is degraded, while keeping per-token metering across the board. That removes the single-provider outage risk without rewriting your client.

The Mixtral 8x7B API pricing comparison ultimately resolves to: Groq for speed, DeepInfra for cheap scale, Together for control. Pick by workload, not by headline rate.