n4n vs Portkey: caching and rate limiting compared

When you’re deciding between gateways, the practical differences in n4n vs Portkey caching rate limiting shape both your bill and your p99. Both sit in front of model providers, but they implement cache scopes and throttle semantics differently enough to matter once you move past prototype traffic.

Caching architecture

Caching for LLM gateways splits into two problems: avoiding redundant generation and avoiding redundant token spend at the provider. Provider-native caches (Anthropic prompt caching, OpenAI’s system cache) only work if the gateway forwards the right hints.

n4n cache forwarding

n4n passes client-supplied cache-control headers straight to the upstream provider. If you already use Anthropic’s cache-control breakpoints, you keep them:

from openai import OpenAI

client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-...")
client.chat.completions.create(
    model="claude-3-5-sonnet",
    messages=[{"role":"user","content":"Long context..."}],
    extra_headers={"anthropic-cache-control": "max-age=600"}
)

The gateway does not invent its own cache layer; it relies on the provider’s cache and your routing directives. That keeps cache hits consistent with what you’d see calling the provider directly.

Portkey cache layers

Portkey builds its own cache in addition to passing through provider caches. It supports a simple cache keyed on the exact request payload and a semantic cache using embeddings to match near-duplicate prompts. You configure it per virtual key or per call:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.portkey.ai/v1",
    api_key="pk-...",
    default_headers={"x-portkey-virtual-key": "vk-prod"}
)
# Portkey reads cache mode from a header or config block
client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role":"user","content":"What is rate limiting?"}],
    extra_headers={"x-portkey-cache": "simple"}
)

The semantic cache reduces spend on paraphrased queries but adds an embedding round-trip on a miss.

Rate limiting mechanics

Rate limits protect downstream provider quotas and your budget. The two gateways take opposite default postures.

Per-key quotas and overflow

Portkey attaches rate limits to virtual keys. You set RPM and TPM caps in the dashboard; exceeding them returns 429 with a retry-after hint. n4n enforces provider-originated limits and triggers automatic fallback when a provider is rate-limited or degraded, rather than hard-blocking at the gateway layer.

# Portkey virtual key with explicit limit (dashboard equivalent in code)
from portkey_ai import Portkey
portkey = Portkey(
    api_key="pk-...",
    virtual_key="vk-prod",
    config={"rate_limit": {"rpm": 100, "tpm": 50000}}
)

With n4n you express routing preferences and let the gateway shift traffic:

{
  "model": "gpt-4o",
  "routing": {"fallback": ["azure/gpt-4o", "claude-3-5-sonnet"]}
}

Fallback behavior under throttling

n4n’s automatic fallback kicks in on provider 429/5xx without client code changes. Portkey supports fallbacks too, but you typically declare them in the config object. Both approaches work; the difference is whether the gateway assumes responsibility (n4n) or the caller declares policy (Portkey).

Cost model and metering

Portkey bills on a platform subscription plus per-request or per-token overage depending on plan. n4n uses per-token usage metering on every routed call, so cost tracks exactly what the provider charges plus the gateway margin.

If your traffic is bursty, n4n’s token metering avoids paying for idle virtual-key reservations. Portkey’s model can be cheaper if you stay inside a flat-tier request count and lean on its cache to cut provider tokens.

Latency and throughput characteristics

Adding a gateway always adds one network hop. n4n’s single OpenAI-compatible endpoint that addresses 240+ models means one connection pool and no per-provider TLS setup. Portkey’s feature-rich config parsing adds microseconds of overhead but gives finer control.

Caching wins dominate latency math. Portkey’s semantic cache can return in single-digit milliseconds on a hit; n4n’s provider-cache passthrough still rounds to the provider’s cached completion speed, which is faster than generation but not local.

Developer ergonomics

n4n feels like talking to one provider. You point your existing OpenAI client at the endpoint, add routing headers, done. Portkey requires adopting virtual keys and a config schema, but rewards that with dashboard observability and cache tuning.

# n4n: just swap base_url
export OPENAI_BASE_URL=https://api.n4n.ai/v1
export OPENAI_API_KEY=sk-...

# Portkey: need virtual key header in every client
export OPENAI_BASE_URL=https://api.portkey.ai/v1
export OPENAI_API_KEY=pk-...

Ecosystem and integrations

n4n.ai ships one OpenAI-compatible endpoint that addresses 240+ models, which covers most inference needs without extra adapters. Portkey maintains a larger native integration surface: LangChain callbacks, Vertex AI bridges, and a hosted analytics UI.

For teams already standardized on OpenAI SDKs, either works. If you need a managed dashboard with per-key analytics, Portkey invests more there.

Hard limits and quotas

Neither gateway publishes infinite headroom. Portkey enforces account-level monthly request ceilings on free tiers. n4n honors provider quotas and will fallback rather than queue; if all routed providers are over limit, it returns the last provider error.

Comparison table

Which to choose

High-volume cost-sensitive batches

If you run millions of token-heavy jobs and want metering to match provider spend exactly, n4n vs Portkey caching rate limiting favors n4n because per-token billing and automatic fallback keep pipelines moving when one provider throttles.

Latency-critical interactive apps

When p95 matters and users repeat similar prompts, Portkey’s semantic cache cuts tail latency without provider round-trips. The extra config work pays off in reduced generation calls.

Multi-provider redundancy

Both handle failover. Choose n4n if you want the gateway to own fallback silently; choose Portkey if you want explicit control over each virtual key’s quota and routing policy.

Pick based on whether you want the gateway to be a thin, provider-faithful proxy (n4n) or a policy-heavy control plane (Portkey).