A unified llm api definition describes a single programmatic contract that hides the differences between disparate model vendors behind one request/response shape. Instead of writing separate client code for OpenAI, Anthropic, and Google, you send the same JSON to one endpoint and specify the target model by name or routing hint.
What a unified LLM API actually is
The core contract
The de facto standard for a unified llm api definition is the OpenAI chat completions schema. It is a POST to /v1/chat/completions with a model string, a messages array, and sampling parameters like temperature and max_tokens. The response returns choices, usage, and optionally stream deltas.
{
"model": "anthropic/claude-opus-4-8",
"messages": [
{"role": "system", "content": "You are terse."},
{"role": "user", "content": "What is a circuit breaker?"}
],
"max_tokens": 150
}The unified layer maps that shape onto whatever the backend provider expects. Anthropic’s native API wants system as a top-level field and uses max_tokens_to_sample; the gateway translates. Google’s Gemini uses contents and generationConfig; the gateway translates again. Your code never sees those differences.
Routing and model identity
Model names are namespaced strings, not opaque IDs. A robust unified llm api definition specifies a convention like provider/model-family. Examples: openai/gpt-5, anthropic/claude-opus-4-8, google/gemini-3, meta/llama-4. The gateway parses the prefix to select the upstream, then passes the suffix as the provider’s native model tag.
Client routing directives extend this. You can pin a provider, set fallbacks, or express preference for region or price tier via headers or extension fields. The contract must state how those directives are honored and what happens when they conflict with availability.
How it works under the hood
Request normalization
Every provider has opinionated constraints. OpenAI accepts tool_calls inside assistant messages; Anthropic expects tools and a different function-call shape. A correct gateway builds an internal normalized AST for the conversation, then serializes to the target backend. It strips unsupported fields and emits provider-required ones (e.g., Anthropic mandates a tools block even if empty when any tool is referenced).
Response adaptation
Streaming is where most abstractions break. OpenAI streams choices[0].delta; Anthropic streams content_block_delta; Gemini streams candidates[0].content.parts. The unified API replays a normalized SSE stream that matches the OpenAI chunk format, including usage at the end if the provider supports it. Tool call aggregation—where a model emits partial JSON across many chunks—is reconstructed server-side so your parser gets a complete call.
Fallback and degradation
When a provider returns 429 or 503, a mature gateway retries against a configured secondary. Some gateways, including n4n.ai, implement automatic fallback when a provider is rate-limited or degraded, while still exposing per-token usage metering. The key engineering detail: fallback must be opt-in per route, because silently swapping gpt-5 for llama-4 changes output contracts in ways your application may not tolerate.
Why it matters for integration code
Kill the SDK zoo
Maintaining three official SDKs means three release cadences, three auth models, and three sets of type definitions. A unified llm api definition lets you keep one HTTP client. In a TypeScript service, that is one openai package instance pointed at a different baseURL.
import OpenAI from "openai";
const llm = new OpenAI({
baseURL: "https://api.n4n.ai/v1",
apiKey: process.env.LLM_KEY,
});Centralized token metering
Per-token accounting is mandatory for cost control. The unified response surfaces usage.prompt_tokens and usage.completion_tokens regardless of backend. You pipe that into your own analytics without writing per-provider parsers.
resp = client.chat.completions.create(model="google/gemini-3", messages=...)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)Cache control forwarding
Providers now support prompt caching. Anthropic uses cache_control on message blocks; OpenAI uses predicted_outputs and similar hints. A real unified llm api definition forwards these hints untouched via extension fields or reserved headers, so you keep cache hit rates without bypassing the abstraction.
Concrete example: one endpoint, four models
Below is a minimal script that queries four distinct model families through one OpenAI-compatible endpoint. It uses namespaced model strings and prints the first 80 characters of each reply.
from openai import OpenAI
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="sk-your-key",
)
models = [
"openai/gpt-5",
"anthropic/claude-opus-4-8",
"google/gemini-3",
"meta/llama-4",
]
for m in models:
resp = client.chat.completions.create(
model=m,
messages=[{"role": "user", "content": "Explain idempotency keys in APIs."}],
max_tokens=120,
)
text = resp.choices[0].message.content
print(f"{m}: {text[:80]}")If you need to force a specific route, pass a header. The gateway honors client routing directives and forwards provider cache-control hints, so the same call can prefer a region or avoid a degraded path:
resp = client.chat.completions.create(
model="anthropic/claude-opus-4-8",
messages=[{"role": "user", "content": "Draft a SQL migration."}],
extra_headers={"x-n4n-routing": "prefer:us-east,fallback:openai/gpt-5"},
)Common misconceptions
“It’s just a dumb proxy”
A proxy forwards bytes. A unified API transforms request and response shapes, reconciles streaming protocols, and normalizes error codes. When Anthropic returns error.type: rate_limit_error and OpenAI returns error.code: 429, the gateway maps both to a single 429 with a structured body your retry logic already understands.
“All models are interchangeable”
The unified llm api definition standardizes the transport, not the model behavior. gpt-5 may follow your system prompt strictly; llama-4 may need different instruction formatting. You still maintain model-specific prompt engineering and eval suites. The API removes plumbing, not judgment.
“You lose provider-specific features”
Cache control, JSON mode, and tool calling are forwarded when the contract defines extension points. If a feature has no cross-provider analog (e.g., Anthropic’s long-context document blocks), the gateway exposes it via pass-through fields rather than dropping it.
“Latency overhead is unacceptable”
A well-built gateway adds single-digit milliseconds for request translation because it is a stateless transform plus a connection pool. The dominant latency is the model inference itself. Measure before assuming the edge is the bottleneck.
When a unified API is the wrong call
If you are building a single-model demo that will never change providers, direct SDK integration is simpler. If you depend on a provider’s experimental endpoint with no cross-vendor equivalent (custom fine-tune training pipelines, proprietary embedding spaces), the abstraction leaks. And if you operate under strict data residency that forbids a middle tier, you must call providers directly.
For most production systems integrating GPT-5, Claude Opus 4.8, Gemini 3, and Llama 4 behind one service, the unified llm api definition is the difference between four brittle integrations and one tested client. You write the retry, the metrics, and the prompt cache logic once. The models change; your code does not.