Llama 4 API rate limits across Groq, Together, Fireworks

The Llama 4 API rate limits Groq Together Fireworks expose are not interchangeable. Groq throttles on a token-bucket keyed to its LPU fleet, Together enforces per-model concurrency on GPU clusters, and Fireworks mixes daily token budgets with per-minute request caps. If you treat them as identical OpenAI-compatible endpoints, your retry logic will fail in production.

How each provider meters Llama 4

Groq

Groq serves Llama 4 Scout and Maverick on its custom LPUs. The free tier is intentionally tight: a small requests-per-minute (RPM) ceiling and a separate tokens-per-minute (TPM) budget. Paid tier lifts both and adds burst headroom. Responses carry x-ratelimit-limit-requests, x-ratelimit-remaining-requests, and token variants. When you exceed them, you get a 429 with retry-after in seconds.

import requests
r = requests.post(
    "https://api.groq.com/openai/v1/chat/completions",
    headers={"Authorization": "Bearer " + KEY},
    json={"model": "llama-4-scout-17b-16e-instruct",
          "messages": [{"role": "user", "content": "ping"}]}
)
print(r.headers.get("x-ratelimit-remaining-tokens"))

Together

Together runs Llama 4 on pooled A100/H100 GPUs. Limits are expressed as concurrent requests per model and a monthly token quota that maps to your plan. There is no hard TPM on the chat endpoint; instead, queuing latency climbs when you saturate concurrency. The x-ratelimit-remaining header reflects slot availability, not tokens.

resp = requests.post(
    "https://api.together.xyz/v1/chat/completions",
    headers={"Authorization": "Bearer " + TOGETHER_KEY},
    json={"model": "meta-llama/Llama-4-Scout-17B-16E-Instruct",
          "messages": [{"role": "user", "content": "ping"}]}
)
print(resp.headers.get("x-ratelimit-remaining"))

Fireworks

Fireworks applies both per-minute request caps and a daily token ceiling that resets at UTC midnight. Their Llama 4 endpoints are OpenAI-compatible but return x-ratelimit-remaining-requests and x-ratelimit-daily-tokens-limit. Exceeding daily tokens returns 429 with a reset timestamp.

curl -s -D - https://api.fireworks.ai/inference/v1/chat/completions \
  -H "Authorization: Bearer $FW_KEY" \
  -d '{"model":"llama4-scout-17b-16e-instruct","messages":[{"role":"user","content":"hi"}]}' \
  -o /dev/null | grep -i ratelimit

Capabilities and model variants

All three host the Llama 4 instruction-tuned checkpoints. Groq initially shipped Scout (17B×16E) with Maverick following; Together and Fireworks mirror the full Meta release set including the larger MoE. Groq’s edge is deterministic low-latency; Together offers fine-tuning and batch inference; Fireworks ships function-calling LoRA adapters alongside base weights. None of them alter the license—you still obey Meta’s acceptable-use policy. The advertised 10M-token context of Scout is exposed on all three, though practical throughput degrades past a few hundred thousand tokens on GPU providers.

Price and cost model

Groq prices per output token with a separate small input fee; free tier is subsidized and not for production. Together uses a per-million-tokens rate that drops with committed volume and adds queue priority for higher tiers. Fireworks blends per-token billing with the daily token allowance as a soft cap—overages bill at a penalty rate. None publish Llama 4 numbers that beat self-hosting at massive scale, but for <100M tokens/month the managed cost is predictable. The Llama 4 API rate limits Groq Together Fireworks attach to these billing models shape your spend ceiling more than the raw rate.

Latency and throughput

Groq is the latency leader: single-digit milliseconds to first token on Scout due to LPU memory bandwidth. Together and Fireworks sit in the 100–400 ms TTFT range on H100s depending on batch size. Throughput per dollar favors Groq for small prompts; Together wins for long-context batch because it exposes max_tokens headroom without strict TPM. Fireworks lands between, with autoscaling that can spike cold pools.

If you want to abstract these shapes, an OpenAI-compatible gateway such as n4n.ai fronts 240+ models and performs automatic fallback when a provider is rate-limited, while still honoring your routing hints and provider cache-control.

Ergonomics

All three implement the OpenAI chat completions schema, so the same client code works with a base_url swap. Groq’s SDK is a thin wrapper; Together adds request_id tracing and streaming stats; Fireworks returns usage with daily_tokens_used. None require custom retry libraries, but you must parse different 429 bodies: Groq gives error.message with code rate_limit_exceeded; Together returns error.type: rate_limit; Fireworks includes error.payload.reset.

async function chat(baseUrl: string, key: string, model: string) {
  const res = await fetch(`${baseUrl}/chat/completions`, {
    method: "POST",
    headers: { "Authorization": `Bearer ${key}`, "Content-Type": "application/json" },
    body: JSON.stringify({ model, messages: [{ role: "user", content: "hi" }] })
  });
  if (res.status === 429) {
    const reset = res.headers.get("retry-after") ?? res.headers.get("x-ratelimit-reset");
    throw new Error(`Rate limited, retry in ${reset}s`);
  }
  return res.json();
}

Ecosystem and tooling

Groq pushes a vibrant community repo set but limited enterprise features. Together provides a model dashboard, fine-tune jobs, and LoRA hosting—useful if Llama 4 needs domain adaptation. Fireworks ships a Python client with built-in caching and prompt templates. For observability, all emit x-request-id; pipe those to your tracing backend. LangChain and Vercel AI SDK have community adapters for each, but you still need to handle the distinct limit headers in your middleware.

Building a resilient client

When you call all three, normalize their limits. Capture remaining requests and tokens, and use a local token bucket to avoid hammering a provider that already sent a 429.

import time, random

class RateLimitError(Exception):
    def __init__(self, retry_after=None):
        self.retry_after = retry_after

def call_with_backoff(fn, max_attempts=5):
    for i in range(max_attempts):
        try:
            return fn()
        except RateLimitError as e:
            wait = e.retry_after or (2**i + random.uniform(0, 1))
            time.sleep(wait)
    raise RuntimeError("exhausted retries")

This pattern works across the Llama 4 API rate limits Groq Together Fireworks because each returns a retry hint. The key is to read the provider-specific header once and feed it into retry_after rather than guessing.

Head-to-head comparison

Which to choose

Real-time user-facing chat: Groq. The Llama 4 API rate limits Groq enforces are the only ones that still let you serve interactive latency at scale if you pay for headroom. Design for TPM bursts and back off on retry-after.

Long-context batch or fine-tuned Llama 4: Together. Its concurrency model avoids token-minute surprises; you queue and consume. Use it when 429 means “queue full” not “budget gone”. Pair with their fine-tune jobs if you need adapter weights.

Cost-bounded startup with daily caps: Fireworks. The daily token ceiling is a natural circuit breaker. If you need to cap spend without building metering, their limit shape does it for you. Watch the UTC reset and shift traffic before midnight if you hit the wall.

Multi-provider production: Don’t pick one. The Llama 4 API rate limits Groq Together Fireworks each fail differently. Put a fallback layer that honors client routing directives and forwards cache-control. That keeps p99 stable when any single provider degrades or you need to route sensitive prompts to a specific region.