How n4n handles model deprecation with automatic routing

When a provider yanks a model from production, your LLM calls start failing unless you have automatic routing around deprecated models in place. This guide lays out a concrete, ordered path to absorb deprecations without emergency code pushes: from inventory to gateway-level fallback. You will ship resilience, not panic.

Inventory every model your code references

Hard-coded model strings hide in config files, prompt templates, and lambda functions. Grep your repo for model= and model: and centralize them in a single JSON document.

{
  "summarizer": "gpt-4-turbo-2024-04-09",
  "classifier": "claude-3-haiku-20240307"
}

If you cannot enumerate the models you call, you cannot track deprecations. Treat this file as a living contract that CI checks on every pull request.

Providers announce sunset dates via mailing lists, status pages, and API GET /models responses that flag deprecated: true. Do not rely on social media. Write a weekly cron that pulls /models from each provider and diffs against your inventory.

curl -s https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_KEY" | jq '.data[] | select(.deprecated)'

Capture the output and open a ticket before the shutdown date. Automate the diff so a missing entry fails the build.

Build a model alias map

Map each deprecated model to a successor that preserves capabilities. Include a fallback chain, not a single replacement, because the successor may also be throttled or retired.

{
  "gpt-4-turbo-2024-04-09": {
    "primary": "gpt-4o-2024-05-13",
    "fallback": ["gpt-4-turbo-2024-07-18", "gpt-4-0125-preview"]
  }
}

Write a resolver function that returns the active model given today’s date and live status.

def resolve_model(alias: str, deprecated_map: dict) -> str:
    entry = deprecated_map.get(alias)
    if not entry:
        return alias
    return entry["primary"]

Keep this map in a config store you can update without redeploying binaries.

Implement client-side fallback as a first line of defense

Automatic routing around deprecated models should not be the only safety net. Wrap your completion call in a retry loop that walks the fallback chain on 404 or model_deprecated errors.

from openai import OpenAI, APIError

client = OpenAI(api_key="sk-...")

def chat_with_fallback(model_chain, messages):
    for model in model_chain:
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except APIError as e:
            if e.status_code in (404, 400) and "deprecated" in str(e):
                continue
            raise
    raise RuntimeError("All models in chain unavailable")

This catches the case where your inventory lags behind the provider. It also lets you fail loudly if the entire chain is dead.

Offload resilience to gateway-level automatic routing around deprecated models

Client-side chains add latency and complexity. A better architecture pushes the problem to an inference gateway. An OpenAI-compatible endpoint that addresses 240+ models can intercept a request to a retired model and redirect it to a compatible replacement without client changes.

n4n.ai operates such a gateway: when a provider is rate-limited or degrades, it performs automatic fallback, and it honors client routing directives and forwards provider cache-control hints. Your code sends the old model name; the gateway returns a valid response from a live substitute.

curl https://api.n4n.ai/v1/chat/completions \
  -H "Authorization: Bearer $N4N_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4-turbo-2024-04-09","messages":[{"role":"user","content":"Hi"}]}'

If that model is deprecated, the gateway’s automatic routing around deprecated models selects a configured successor and returns it with the actual model name in the response model field. You get per-token usage metering on the actual model used. The client code stays unchanged; the deprecation is absorbed at the edge.

The tradeoff: you must trust the gateway’s capability matching. Validate it (see below).

Validate with shadow traffic and contract tests

Before depending on silent routing, send duplicate requests to the deprecated model and its successor in a shadow pipeline. Compare outputs for your key tasks: JSON validity, length, tone.

def shadow_compare(prompt, old_model, new_model):
    old = client.chat.completions.create(model=old_model, messages=prompt)
    new = client.chat.completions.create(model=new_model, messages=prompt)
    assert json.loads(new.choices[0].message.content)  # contract

If the successor breaks your parse logic, adjust prompts or pick a different fallback. Run this as a nightly job against a sample of production prompts.

Monitor fallback rate and latency

Emit a counter every time a request hits a deprecated alias. In Prometheus terms:

- name: deprecated_model_fallbacks_total
  help: Count of requests routed away from deprecated models

Alert if the rate exceeds 1% of traffic for a given model—that means clients still reference dead strings. Also track p95 latency; automatic routing around deprecated models adds a lookup, but it should be sub-millisecond at the gateway. If latency spikes, your fallback chain is hitting live provider errors.

Common pitfalls

Capability drift: a newer model may reject a system prompt format your code assumes. Prompt injection guards differ. Version pinning: if you pin gpt-4-turbo without date, you may already be on a deprecated build. Cache hints: provider cache-control headers are forwarded, but a routed model may not support the same cache namespace, silently dropping hits.

Another pitfall is assuming the fallback model has identical token limits. A summarizer tuned for 32k context may route to an 8k model and truncate. Encode context size in your alias map.

Tradeoffs

Gateway routing trades a small amount of observability for resilience. You no longer control exactly which model answered, so log the model field returned in the response. There is a risk of silent quality change if the successor is weaker at a niche task. Mitigate with periodic shadow evals and by keeping the alias map reviewed by a human before a scheduled deprecation.

Build the inventory, subscribe to signals, map aliases, code client fallback, then let the gateway handle the long tail. That ordered path keeps your LLM features online when providers pull the rug.