LangChain model_name overrides for OpenAI-compatible gateways

When you point LangChain’s ChatOpenAI client at an OpenAI-compatible endpoint, the langchain model_name override gateway logic decides which backend model actually receives your request. Misconfigure it and you’ll either get hard 404s or silently fall back to an unintended default. This how-to gives you exact steps to override model names cleanly and verify the wire payload.

Step 1: Install the right package and construct the client

LangChain split OpenAI support into langchain-openai. If you’re still on the monolithic langchain package, upgrade. The client takes base_url, api_key, and a model identifier.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://gateway.example.com/v1",
    api_key="sk-your-gateway-key",
    model="openai/gpt-4o",  # gateway-specific model id
    temperature=0.2,
)

model vs model_name

Recent langchain-openai versions renamed model_name to model. The old param still works as a deprecated alias but emits warnings. For any new langchain model_name override gateway code, use model=. The value you pass is forwarded verbatim as the model field in the Chat Completions request body.

Step 2: Understand what model_name becomes on the wire

LangChain does not transform the string. It serializes it directly into the JSON payload sent to {base_url}/chat/completions. A gateway that exposes 240+ models expects its own catalog IDs, not the raw OpenAI string.

curl https://gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer sk-your-gateway-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4o","messages":[{"role":"user","content":"hi"}]}'

If you set model="gpt-4o" but the gateway only knows openai/gpt-4o, you’ll get a 400 or a fallback to a default. The langchain model_name override gateway mapping is strictly string-based unless the gateway itself does translation.

Step 3: Override model_name with environment configuration

Hard-coding model IDs in source is a deployment smell. Read from the environment so you can shift models per environment without a rebuild.

import os
from langchain_openai import ChatOpenAI

MODEL = os.getenv("APP_LLM_MODEL", "anthropic/claude-3-5-sonnet")
BASE = os.getenv("APP_LLM_BASE", "https://gateway.example.com/v1")
KEY = os.getenv("APP_LLM_KEY", "sk-your-gateway-key")

llm = ChatOpenAI(base_url=BASE, api_key=KEY, model=MODEL)

This env-var pattern is the most common langchain model_name override gateway approach for staged deployments. Set APP_LLM_MODEL=meta-llama/llama-3-70b in staging, openai/gpt-4o in prod, and the same binary routes correctly.

Step 4: Swap models at runtime without rebuilding the client

If your app needs to pick a model per request (e.g., user selects “fast” vs “smart”), don’t mutate a shared instance. Construct a fresh client per call or wrap it in a factory. LangChain clients are cheap to instantiate.

def make_llm(model: str) -> ChatOpenAI:
    return ChatOpenAI(
        base_url="https://gateway.example.com/v1",
        api_key="sk-your-gateway-key",
        model=model,
    )

# In your handler
def handle(prompt: str, tier: str):
    model_id = "openai/gpt-4o-mini" if tier == "fast" else "anthropic/claude-3-opus"
    return make_llm(model_id).invoke(prompt)

Trying to override via llm.bind(model="x") does not reliably replace the top-level model field in current langchain-openai releases; it merges into model_kwargs and can produce an invalid body. Use the factory.

Step 5: Send gateway-specific routing and cache directives

Many gateways accept extra JSON keys to control provider selection, fallback, or caching. LangChain forwards unknown keys via model_kwargs.

llm = ChatOpenAI(
    base_url="https://gateway.example.com/v1",
    api_key="sk-your-gateway-key",
    model="anthropic/claude-3-opus",
    model_kwargs={
        "provider": "anthropic",
        "cache_control": {"type": "ephemeral", "ttl": "3600"},
    },
)

Gateways such as n4n.ai honor client routing directives and forward provider cache-control hints, so the model_name you send must match their catalog exactly and the extra kwargs must be permitted. If you pass a model string the gateway doesn’t recognize, the routing hint is moot. Always validate the model ID against the gateway’s /v1/models endpoint before shipping.

Step 6: Verify the override with an intercepted request

Don’t trust prints of the Python object—inspect the actual HTTP body. The responses library lets you stub the gateway and assert the sent model.

import json
import responses
from langchain_openai import ChatOpenAI

@responses.activate
def test_override():
    responses.add(
        responses.POST,
        "https://gateway.example.com/v1/chat/completions",
        json={
            "id": "1",
            "choices": [{"message": {"role": "assistant", "content": "ok"}}],
            "model": "test/model",
        },
        status=200,
    )
    llm = ChatOpenAI(
        base_url="https://gateway.example.com/v1",
        api_key="x",
        model="test/model",
    )
    llm.invoke("hello")
    body = json.loads(responses.calls[0].request.body)
    assert body["model"] == "test/model", body
    print("Verified model on wire:", body["model"])

test_override()

Alternatively, set langchain.debug = True before invocation; the debug log dumps the request payload including the model field. Either method confirms your langchain model_name override gateway configuration is correct before you depend on it in production.

Step 7: Avoid the common pitfalls that break overrides

Deprecated param warnings. If you see model_name deprecation warnings, switch to model=. The alias will eventually be removed.

Trailing whitespace or case mismatch. Gateway model catalogs are exact strings. "OpenAI/gpt-4o" is not "openai/gpt-4o".

Default temperature and max_tokens. LangChain sends defaults even if you don’t set them. Gateways ignore or clamp them, but they don’t affect model selection.

Streaming. When you call stream=True, the model field is sent identically. Verify once with streaming on if you use it, because some proxies parse the body differently.

Multiple gateways. If you use more than one base URL (e.g., direct OpenAI for one path, gateway for another), instantiate separate clients. Reusing a client with a patched base_url via monkey-patching leads to subtle cross-contamination.

Model kwargs collisions. Don’t put model inside model_kwargs. It will either be ignored or cause a schema error. Keep model at the top level.

Following these steps gives you a deterministic, observable langchain model_name override gateway setup: the right model ID leaves your process, the gateway routes it, and you can prove it with a test.