Migrating from OpenRouter to n4n without downtime

You can migrate from OpenRouter to n4n zero downtime if you treat the cutover as a routing problem rather than a rewrite. Both gateways implement the OpenAI Chat Completions schema, so your existing SDK calls need only a base URL and credential change.

Step 1: Audit your current OpenRouter integration

Pull every place your code constructs an OpenAI client or hits the /v1/chat/completions endpoint. Capture the base URL, model strings, request params (temperature, max_tokens, stop), and any custom headers like HTTP-Referer or X-Title that OpenRouter uses for app attribution.

What to log

Record the exact JSON payloads for a representative sample of traffic. You need this baseline to confirm the new gateway returns equivalent completions. Pay special attention to model fields: OpenRouter often prefixes with a vendor slug (anthropic/claude-3.5-sonnet), and those strings must remain valid on the destination.

from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-...",
)
resp = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[{"role": "user", "content": "ping"}],
    extra_headers={"HTTP-Referer": "https://myapp.com", "X-Title": "MyApp"},
)
print(resp.model_dump_json())

Store these dumps in a structured log or a sidecar table. They become your regression set.

Step 2: Stand up a shadow mode dual-write

Before flipping any switches, send production requests to both gateways and compare outputs. Wrap your client in a function that fires to OpenRouter and to the n4n endpoint concurrently, logs both responses, and returns the OpenRouter result to users. n4n.ai exposes a single OpenAI-compatible endpoint covering 240+ models, so you can keep the same model identifiers.

import asyncio
from openai import OpenAI

or_client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key="sk-or-...")
n4n_client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-n4n-...")

async def shadow_chat(model, messages):
    or_task = asyncio.to_thread(or_client.chat.completions.create,
        model=model, messages=messages)
    n4n_task = asyncio.to_thread(n4n_client.chat.completions.create,
        model=model, messages=messages)
    or_resp, n4n_resp = await asyncio.gather(or_task, n4n_task)
    # log diffs: choices, finish_reason, usage
    return or_resp

Comparing responses

Run this for a few hours. If the n4n_resp.choices[0].message.content diverges wildly on identical prompts, investigate model version pinning before proceeding. Use a deterministic seed where the model supports it to reduce noise. Track finish_reason mismatches—they signal different safety or truncation behavior.

Step 3: Swap credentials in staging

Create a staging environment that points entirely at the new gateway. Use environment variables so the same binary runs in both modes.

export LLM_BASE_URL="https://api.n4n.ai/v1"
export LLM_API_KEY="sk-n4n-..."

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
)

Run your full test suite against staging. Pay attention to provider-specific extensions: OpenRouter passes error.code values that may differ. Normalize error handling so your retry logic keys off HTTP status, not vendor error strings.

Error normalization

Map 429, 500, 503 to your retry queue regardless of the body. Write a thin wrapper:

def safe_create(client, **kwargs):
    try:
        return client.chat.completions.create(**kwargs)
    except Exception as e:
        if e.status_code in (429, 500, 503):
            raise TransientLLMError(e)
        raise

Step 4: Honor routing directives and cache hints

The gateway honors client routing directives and forwards provider cache-control hints. If your system sends cache_control in the messages to trigger prompt caching on Anthropic, keep it; it will be passed through. You can also force a specific provider via headers if you need to avoid a degraded path during cutover.

resp = client.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",
    messages=[
        {"role": "user", "content": "System: cache this"},
        {"role": "user", "content": "What is 2+2?",
         "extra_body": {"cache_control": {"type": "ephemeral"}}},
    ],
    extra_headers={"X-Route-Prefer": "anthropic"},
)

Automatic fallback engages when a provider is rate-limited or degraded, but explicit routing helps you isolate problems during migration. Do not rely on fallback as a crutch; treat it as a safety net while you validate primary routes.

Step 5: Canary production traffic

Use a weighted router in your service mesh or a simple in-process flag to send a small percentage of live calls to n4n. Start at 5%, watch error rates, then bump to 25%, 50%, 100%.

import random
from openai import OpenAI

or_client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key="sk-or-...")
n4n_client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-n4n-...")

def routed_chat(model, messages, canary_pct=5):
    if random.randint(1, 100) <= canary_pct:
        return n4n_client.chat.completions.create(model=model, messages=messages)
    return or_client.chat.completions.create(model=model, messages=messages)

Monitoring

Track latency percentiles per route. If p95 latency on the new path exceeds your SLO by more than 20%, hold the rollout. Export token counts per route to a metrics backend:

statsd.incr("llm.tokens", resp.usage.total_tokens, tags=[f"route:{route}"])

Step 6: Cut over and decommission

When canary hits 100% with clean metrics, remove the OpenRouter branch. Delete the old client instantiation and the shadow logger. Rotate the OpenRouter key to revoke access.

# Final state: only n4n
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-n4n-...")

Keep the dual-client code in git history, not in production. Document the cutover time in your incident log as a completed migration, not an outage.

Verifying success

Success means zero increase in 5xx rates and no customer-visible completion changes. Pull token counts from responses: n4n.ai provides per-token usage metering, so you can reconcile usage.completion_tokens against your billing dashboard directly.

{
  "id": "chatcmpl-123",
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 34,
    "total_tokens": 46
  }
}

Add an alert on usage.total_tokens drift between old and new baselines during the canary phase. If numbers match within 1% on identical prompts, the migrate from OpenRouter to n4n zero downtime objective is met.

Caveats

Model aliases sometimes differ. OpenRouter uses openai/gpt-4o while some gateways prefer gpt-4o. The n4n endpoint keeps OpenAI-compatible naming, but verify each model ID against its model list. Private or fine-tuned models hosted only on OpenRouter will not exist on the new path; you must either replicate the fine-tune or keep a narrow OpenRouter route for those specific IDs.

Streaming behaves identically at the protocol level, but client-side buffer timeouts may need adjustment if edge latency changes. Test stream=True under load before canary.

If you follow these steps, the migrate from OpenRouter to n4n zero downtime plan becomes a configuration change backed by measurement, not a leap of faith.