n4nAI
Open console →
Framework tutorialsHow-toLangChain Expression Language (LCEL) Chains

LCEL with n4n.ai: routing between GPT-4o and Llama 3.3

Step-by-step guide to implementing lcel n4n.ai model routing gpt-4o llama with LangChain Expression Language for cost-efficient and resilient LLM inference.

n4n Team3 min read642 words

Audio narration

Coming soon — every post will get a voice note here.

Building lcel n4n.ai model routing gpt-4o llama lets you send simple classification prompts to Llama 3.3 and escalate to GPT-4o only when the task needs stronger reasoning. This walkthrough shows how to assemble that logic with LangChain Expression Language (LCEL) and an OpenAI-compatible gateway. You’ll end up with a single chain that picks the right model per request without changing caller code.

Step 1: Install dependencies and configure the endpoint

Install the minimal LangChain packages. You do not need the full langchain metapackage.

pip install langchain-openai langchain-core python-dotenv

Set your API key for the gateway. The endpoint is OpenAI-compatible, so ChatOpenAI works unchanged aside from base_url:

import os
from dotenv import load_dotenv

load_dotenv()
os.environ["N4N_API_KEY"] = os.getenv("N4N_API_KEY", "")
BASE_URL = "https://api.n4n.ai/v1"

Keep the key out of source control. The gateway fronts 240+ models, but we will pin two.

Step 2: Instantiate the two chat models

Use explicit model strings. On OpenRouter-class gateways the namespace typically prefixes the provider. Adjust if your deployment uses different slugs.

from langchain_openai import ChatOpenAI

llama = ChatOpenAI(
    model="meta-llama/llama-3.3-70b-instruct",
    temperature=0,
    base_url=BASE_URL,
    api_key=os.environ["N4N_API_KEY"],
    max_tokens=1024,
)

gpt4o = ChatOpenAI(
    model="openai/gpt-4o",
    temperature=0,
    base_url=BASE_URL,
    api_key=os.environ["N4N_API_KEY"],
    max_tokens=1024,
)

Set temperature=0 for deterministic routing tests. In production you may relax that per model.

Step 3: Define routing predicates

Routing logic should be cheap and side-effect free. A common pattern: use Llama for short, factual prompts and GPT-4o for anything requiring multi-step analysis.

def needs_powerful_model(x: dict) -> bool:
    prompt = x.get("prompt", "")
    if not isinstance(prompt, str):
        prompt = str(prompt)
    # Escalate on explicit ask or length.
    if "analyze" in prompt.lower() or "reason" in prompt.lower():
        return True
    if len(prompt.split()) > 40:
        return True
    return False

Why heuristic routing instead of an LLM judge

A second model call to decide routing adds latency and cost. For most workloads a regex or token-count check captures 90% of the value. Reserve learned routers for when you have clear failure data.

Step 4: Compose the branch with LCEL

RunnableBranch is the native LCEL primitive for conditional execution. The first tuple whose predicate returns True wins; the final positional argument is the default.

from langchain_core.runnables import RunnableBranch, RunnablePassthrough

router = RunnableBranch(
    (needs_powerful_model, gpt4o),
    llama,
)

chain = RunnablePassthrough.assign(model=RunnableLambda(lambda x: "gpt-4o" if needs_powerful_model(x) else "llama-3.3")) | router

The RunnablePassthrough.assign step annotates which model will handle the request. This is useful for logging without parsing the response.

Handling structured input

If your caller passes a dict with more fields, keep the branch signature consistent:

def route_fn(x: dict) -> ChatOpenAI:
    return gpt4o if needs_powerful_model(x) else llama

router = RunnableBranch(
    (lambda x: needs_powerful_model(x), gpt4o),
    llama,
)

RunnableBranch accepts callables directly; wrapping in lambdas is optional.

Step 5: Add streaming and metadata passthrough

LCEL chains stream if every link supports it. Both ChatOpenAI instances do. Wrap the chain in .with_config({"callbacks": [...]}) or just .stream():

for chunk in chain.stream({"prompt": "Summarize the RFC briefly."}):
    print(chunk.content, end="", flush=True)

The gateway returns provider response metadata. Access it after invocation:

resp = chain.invoke({"prompt": "What is the capital of France?"})
print(resp.response_metadata.get("model"))
print(resp.response_metadata.get("usage"))

Per-token usage appears in usage. The gateway meters tokens accurately even when it forwards provider cache-control hints, so your billing matches what you see in the response.

Step 6: Execute and verify success

Run two contrasting prompts and confirm the branch selects different models.

simple = chain.invoke({"prompt": "What is 2+2?"})
complex_p = chain.invoke({"prompt": "Analyze the geopolitical implications of the recent oil supply shift over the last decade in detail, including secondary economic effects."})

print("Simple model:", simple.response_metadata.get("model"))
print("Complex model:", complex_p.response_metadata.get("model"))

Verification checklist:

  • The simple prompt resolves to meta-llama/llama-3.3-70b-instruct (or your slug).
  • The complex prompt resolves to openai/gpt-4o.
  • Both responses contain usage.prompt_tokens and usage.completion_tokens greater than zero.
  • Streaming yields incremental AIMessageChunk objects without errors.

If both return the same model, your predicate is too loose or too strict—adjust the word threshold.

Step 7: Production hardening

The core of lcel n4n.ai model routing gpt-4o llama is now a single Runnable you can drop into a FastAPI app or a LangServe endpoint. A few notes from shipping this:

Timeouts: Set request_timeout on ChatOpenAI (e.g., 30s). LCEL propagates exceptions; wrap with RunnableRetry if you want automatic retries on transient 5xx.

Fallback: The gateway automatically falls back when a provider is rate-limited or degraded, but that is separate from your branch. Your branch picks the primary model; the gateway handles provider-level resilience. Do not build your own provider ping logic.

Cache control: Forward cache hints by passing extra_body={"cache": {"type": "ephemeral"}} to ChatOpenAI if your workload repeats system prompts. The gateway honors client routing directives and forwards those hints.

Observability: Log the model annotation from Step 4 alongside the response latency. Over a week you will see exactly how often GPT-4o is invoked—typically far less than 20% on support-style traffic.

Closing implementation note

You now have a composable, testable routing layer. Because everything is LCEL, you can swap RunnableBranch for a RunnableParallel that calls both models and picks the better answer later, or insert a RunnableLambda that rewrites prompts per model. The routing decision stays declarative and the caller sends one dict.

To extend, add more branches: a small embedding model for semantic routing, or a fine-tuned classifier. The pattern does not change.

Tagslangchainlceln4n-aimodel-routing

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Framework tutorialsDebugging LCEL chains with LangSmith tracingLearn to debug LangChain Expression Language chains using LangSmith tracing with step-by-step setup, runnable code, and verification techniques.Framework tutorialsLCEL RunnableLambda: adding custom functions to chainsLearn to wrap custom Python functions as LCEL RunnableLambda components for composable, streaming-capable LangChain pipelines.Framework tutorialsParallel execution in LangChain using LCELLearn to run independent LCEL chains in parallel using RunnableParallel, RunnablePassthrough, and async patterns to cut latency in production LangChain applications.Framework tutorialsError handling and fallbacks in LCEL chainsPractical guide to implementing robust error handling, retries, and fallback strategies in LangChain Expression Language chains for production LLM applications.Framework tutorialsLCEL RunnableBranch: conditional logic in chainsLCEL RunnableBranch lets you route inputs to different chains based on conditions — here's how it works, when to use it, and a production-ready example.Framework tutorialsLCEL RunnableParallel for multi-step LangChain pipelinesBuild multi-step LangChain pipelines with LCEL RunnableParallel — parallel branches, shared state, error handling, and verification steps.

All langchain expression language (lcel) chains posts →