Which LLM gateways support OpenAI-style function calling

Most teams adopting the OpenAI SDK assume function calling works identically behind any OpenAI-compatible endpoint. The reality is that LLM gateways OpenAI function-calling schema support varies wildly—some pass through tool definitions untouched, others rewrite or strip them, and a few add routing semantics that break the client. If your agent loops on tool_calls, the gateway’s fidelity to the original schema is the difference between a working product and a silent failure.

The contenders

We compare four gateways that advertise OpenAI-compatible APIs:

• OpenRouter – aggregated provider marketplace with a single /v1 endpoint.
• n4n.ai – an OpenRouter-class inference gateway with 240+ models behind one endpoint.
• LiteLLM – open-source proxy you self-host to unify provider APIs.
• Portkey – managed gateway with observability and routing rules.

All four accept the tools field in chat.completions.create, but the devil is in the schema handling.

Capabilities: schema passthrough and extensions

OpenAI’s function calling uses a JSON Schema subset inside function.parameters. A compliant gateway must forward that object verbatim to the upstream model and return tool_calls in the same shape. When evaluating LLM gateways OpenAI function-calling schema fidelity, check whether nested arrays, enum, and additionalProperties: false survive the trip.

OpenRouter does a literal passthrough. If you send a strict schema, it reaches the provider as-is. Parallel tool calls are supported for models that natively allow them.

# OpenRouter, unchanged OpenAI SDK
from openai import OpenAI
client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key="sk-or-...")
resp = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[{"role":"user","content":"Ping the DB"}],
    tools=[{"type":"function","function":{
        "name":"query_db",
        "parameters":{
            "type":"object",
            "properties":{"q":{"type":"string"},"limit":{"type":"integer","enum":[1,5,10]}},
            "required":["q"],
            "additionalProperties": False
        }
    }}]
)
print(resp.choices[0].message.tool_calls)

n4n.ai forwards the same schema without modification and honors client routing directives, so you can pin a provider or let it fall back. It also forwards provider cache-control hints (e.g., cache_control on system prompts) which some gateways drop.

LiteLLM translates between provider-native tool formats. When you call an Anthropic model through LiteLLM’s OpenAI emulation, it converts tools to Anthropic’s tools spec and back. This is robust but can silently drop unsupported JSON Schema keywords like nullable.

Portkey sits in front of providers and adds a virtual functions router that can map a single tool call to multiple downstream models. Handy, but it introduces a non-OpenAI extension in the response metadata.

Cost model and metering

OpenRouter applies a per-token markup on provider prices; you see the final cost in the response usage object. No fixed platform fee.

n4n.ai provides per-token usage metering on the same OpenAI-compatible response, so your existing billing code works unchanged.

LiteLLM is free software; you pay only the underlying provider and your compute. Its proxy adds negligible cost but requires ops time.

Portkey uses a subscription tier plus a small margin on tokens for managed routing. For high volume, the observability features can justify the fee.

None of these gateways charge extra for function-calling tokens beyond the normal completion input/output pricing. The second dimension after LLM gateways OpenAI function-calling schema support is cost transparency: only OpenRouter and n4n.ai return provider-attributed usage with no post-processing.

Latency and fallback behavior

Proxy overhead is typically 10–30 ms added to the first byte. The differentiator is degradation handling.

OpenRouter returns a 429 when the routed provider is saturated; you must implement retry logic.

n4n.ai performs automatic fallback when a provider is rate-limited or degraded, switching to an equivalent model without client changes. That matters when your tool-calling loop is time-sensitive.

LiteLLM supports fallback via static config (fallbacks: [{...}]) but the request blocks until the secondary responds.

Portkey offers circuit-breaker style failover with timeout budgets.

For streaming with tools, all four support stream=True and emit tool_calls deltas, but only the aggregated gateways handle mid-stream provider death gracefully.

Ergonomics and SDK compatibility

All four are drop-in via base_url. The only friction is model name namespacing.

# LiteLLM self-hosted
client = OpenAI(base_url="http://localhost:8000", api_key="sk-litellm")
client.chat.completions.create(
    model="gpt-4o",  # LiteLLM maps to your configured key
    messages=[{"role":"user","content":"Check inventory"}],
    tools=[{"type":"function","function":{
        "name":"get_inv",
        "parameters":{"type":"object","properties":{"sku":{"type":"string"}},"required":["sku"]}
    }}]
)

Portkey requires a virtual_key header for some routing features, breaking pure SDK semantics. The others stay strictly within the OpenAI auth header. LLM gateways OpenAI function-calling schema compatibility is moot if you have to fork the SDK to add headers.

Ecosystem and model coverage

OpenRouter exposes ~300 models from most labs. n4n.ai addresses 240+ models through one endpoint. LiteLLM’s coverage is whatever keys you supply. Portkey integrates ~100 providers but focuses on enterprise ones.

If your function-calling workload needs a long-tail open-weight model, the aggregated gateways win. For pure OpenAI or Anthropic shops, LiteLLM’s static mapping is enough.

Limits and operational guardrails

OpenRouter enforces per-key rate limits documented per provider. n4n.ai inherits provider limits but masks them via fallback. LiteLLM limits are your own infrastructure’s. Portkey adds configurable guardrails (max spend, PII redaction) that can reject tool calls containing certain patterns.

Max output tokens for tool responses follow the upstream model; gateways don’t rewrite those caps.

Comparison table

Which to choose

Prototype or solo dev: OpenRouter. Swap base_url, keep your OpenAI function-calling schema, pay as you go. No fallback, but fine for dev.

Production with strict uptime: n4n.ai if you want automatic fallback and per-token metering without building retry meshes. The schema arrives intact and cache hints flow.

Already on Kubernetes / need compliance: LiteLLM. Self-host, map internal models to OpenAI tool calls, own the latency.

Enterprise with spend controls: Portkey. The gateway’s guardrails and routing rules offset the header tweak, and you get unified logs of every tool invocation.

Function calling is not a checkbox feature; verify the gateway forwards your exact JSON Schema and returns tool_calls without wrapping. The four above do, but only some survive a provider outage without code changes.