Comparing API formats: GPT-5, Claude Opus 4.8, Gemini 3, Llama 4

When you compare llm api formats for GPT-5, Claude Opus 4.8, Gemini 3, and Llama 4, the friction is rarely the model output—it is the wire protocol, the SDK opinions, and the rate-limit semantics. Each vendor ships a distinct HTTP shape, and that dictates how much glue code you write before the first token streams.

Wire Protocol and Message Shape

GPT-5 (OpenAI-compatible)

OpenAI’s chat completions schema is the de facto standard. A request is a JSON body with model, messages, temperature, etc. System prompts live inside messages with role: "system".

from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "You are terse."},
        {"role": "user", "content": "Summarize: ..."}
    ],
    stream=True
)

Claude Opus 4.8 (Anthropic)

Anthropic separates the system prompt from messages and uses content blocks for structured input. The endpoint is /v1/messages. System can be a string or an array of text blocks with cache markers.

import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
    model="claude-opus-4-8",
    system=[{"type": "text", "text": "You are terse.",
             "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": "Summarize: ..."}],
    max_tokens=1024,
    stream=True
)

Gemini 3 (Google)

Gemini uses a contents array with parts, and generation params sit under generationConfig. The Python SDK abstracts this but the raw shape differs significantly from the OpenAI style.

import google.generativeai as genai
genai.configure(api_key="...")
model = genai.GenerativeModel("gemini-3")
resp = model.generate_content(
    [{"role": "user", "parts": ["Summarize: ..."]}],
    stream=True
)

Llama 4 (Open weights / hosted)

Most hosted Llama 4 endpoints (Together, Groq, etc.) mirror the OpenAI schema. Self-hosted vLLM also exposes /v1/chat/completions. You call it like GPT-5.

client = OpenAI(base_url="https://your-llama4-host/v1")
resp = client.chat.completions.create(
    model="llama-4-70b",
    messages=[{"role": "user", "content": "Summarize: ..."}]
)

Capabilities and Modality

GPT-5 and Gemini 3 expose native multimodal input (image, audio, video frames) through the same message array. Claude Opus 4.8 supports image input via content blocks but treats audio differently. Llama 4, as open weights, is text-only unless you bolt on a vision encoder; hosted variants may add modalities.

Tool calling is standardized on JSON schema in OpenAI and Anthropic; Gemini uses function_declarations. If you compare llm api formats for tool use, OpenAI and Anthropic are closest in ergonomics, while Gemini requires mapping your schema into its declaration shape.

Cost Model and Metering

OpenAI, Anthropic, and Google bill per token with separate input/output rates; all three offer cached input discounts if you send cache_control markers. Llama 4 is open weights: you pay for GPU hours if self-hosted, or per-token via a hosting provider. There is no universal price—cost depends on your infra or your vendor’s margin.

A gateway that performs per-token usage metering across all four lets you attribute spend without building four billing pipelines.

Latency and Throughput

Quantitative claims here are provider-specific and tier-specific. Generally, Gemini 3’s serving stack is optimized for high batch throughput on TPUs; Claude Opus 4.8 prioritizes long-context coherence with moderate token rates; GPT-5 sits in the middle; Llama 4 latency is a function of your host’s hardware—on H100s it competes, on CPUs it does not.

Streaming is supported by all four, but the SSE event shape differs: OpenAI sends choices[0].delta, Anthropic sends content_block_delta, Gemini sends candidates[0].content.parts. Time-to-first-token varies with context length; million-token prompts on Claude or Gemini incur a noticeable prefill cost that you must budget for.

Error Shapes and Retry Semantics

OpenAI returns error.message with type and code. Anthropic uses error.type like rate_limit_error. Gemini returns error.code (gRPC style) and message. Llama 4 hosts mimic OpenAI. When you compare llm api formats, the retry logic must branch on these shapes unless you normalize.

// OpenAI
{"error":{"message":"Rate limit","type":"rate_limit_error","code":"429"}}
// Anthropic
{"error":{"type":"rate_limit_error","message":"Rate limited"}}
// Gemini
{"error":{"code":429,"message":"Resource exhausted"}}

Backoff strategy should be exponential for 429s across all, but only Anthropic and OpenAI document specific retry-after headers consistently. Gemini sometimes returns 429 without a header, forcing client-side jitter.

Ergonomics and SDK Support

OpenAI’s SDK is the most mature; every language has a wrapper. Anthropic’s SDK is clean but forces a separate system param and content-block lists. Gemini’s SDK rearranges your prompt into parts, which annoys if you migrate from OpenAI. Llama 4’s OpenAI compatibility means zero new code if you already use that client.

If you route through a single OpenAI-compatible gateway such as n4n.ai, you get one endpoint for 240+ models with automatic fallback when a provider is degraded, and it forwards cache-control hints so you keep provider discounts. That collapses the compare llm api formats problem into one client.

Ecosystem and Tooling

GPT-5 has the largest plugin and eval ecosystem. Claude Opus 4.8 is favored for long-document analysis and has first-class prompt caching docs. Gemini 3 integrates with Vertex AI and BigQuery. Llama 4 benefits from HuggingFace weights, LoRA fine-tuning, and self-host flexibility.

For observability, OpenAI and Anthropic have native logging dashboards; Gemini ties into Cloud Monitoring; Llama 4 requires you to stand up your own Prometheus or use the host’s tooling.

Limits and Quotas

Claude Opus 4.8 and Gemini 3 advertise million-token contexts; GPT-5 sits at 128K–256K depending on tier; Llama 4 self-hosted defaults to 128K but can be extended with RoPE scaling. Max output tokens are typically 4K–8K for Claude, 8K+ for Gemini, similar for GPT-5. Rate limits are requests-per-minute budgets that vary by account tier, not by model architecture.

Comparison Table

Which to Choose

Prototype speed: Use GPT-5 or Llama 4 via an OpenAI-compatible client. You write one code path, and every existing LLM library works.

Long-document analysis: Claude Opus 4.8 or Gemini 3. Both handle million-token inputs; pick Claude if you want prompt caching on long system prompts, Gemini if you live in Google Cloud.

Cost-sensitive at scale: Llama 4 self-hosted. You trade ops burden for zero per-token fee. If you lack GPU capacity, use a hosted Llama 4 and compare llm api formats to see which provider gives best throughput.

Multimodal product: Gemini 3 or GPT-5. Gemini’s TPU stack streams video frames cheaply; GPT-5 has broader third-party tooling.

Regulated / on-prem: Llama 4. Only open-weight option here; you control the weights and the data path.

When you compare llm api formats at the end of the day, the decision is less about which model is best and more about which protocol your stack can absorb without rewrites.