Mix GPT-4o and Claude 3.5 Sonnet per agent via n4n.ai

Shipping a multi-agent workflow where the planner uses GPT-4o and the writer uses Claude 3.5 Sonnet normally forces you to manage two API clients and two key sets. To autogen mix gpt-4o claude 3.5 sonnet agents n4n.ai, point every AutoGen agent at a single OpenAI-compatible base URL and set a different model per agent. The gateway handles provider auth, automatic fallback on rate limits, and per-token metering behind that one endpoint.

Step 1: Provision access and set environment variables

n4n.ai exposes a single OpenAI-compatible endpoint (https://api.n4n.ai/v1) that addresses 240+ models. Export your key and base URL:

export N4N_API_KEY="sk-your-key"
export N4N_BASE_URL="https://api.n4n.ai/v1"

Do not commit the key. The base URL has no trailing slash; the OpenAI SDK appends /chat/completions automatically.

Step 2: Install AutoGen and lock the version

Use Python 3.10+. Install pyautogen:

pip install "pyautogen==0.2.32"

Older versions lack stable llm_config passthrough for extra_headers. If you use a different version, verify that autogen.AssistantAgent accepts arbitrary keys in llm_config and forwards them to openai.ChatCompletion.create.

Step 3: Build a per-agent LLM config factory

AutoGen sends llm_config directly to the OpenAI client. Vary only model and keep the rest constant. Add extra_headers to pass cache-control hints; the gateway forwards them to the upstream provider.

import os

def make_llm_config(model: str) -> dict:
    return {
        "model": model,
        "api_key": os.environ["N4N_API_KEY"],
        "base_url": os.environ["N4N_BASE_URL"],
        "temperature": 0.2,
        "max_tokens": 1024,
        "timeout": 30,
        "extra_headers": {
            "cache-control": "max-age=300",
        },
    }

The timeout prevents a hung agent from blocking the group chat. The cache-control header tells the gateway to honor edge caching for identical prompts—useful when the planner repeats similar decomposition calls.

Step 4: Define agents with distinct models

Create a Planner on openai/gpt-4o and a Drafter on anthropic/claude-3.5-sonnet. System messages should be role-specific and short.

import autogen

planner = autogen.AssistantAgent(
    name="Planner",
    llm_config=make_llm_config("openai/gpt-4o"),
    system_message="Decompose user requests into numbered steps. No prose.",
)

drafter = autogen.AssistantAgent(
    name="Drafter",
    llm_config=make_llm_config("anthropic/claude-3.5-sonnet"),
    system_message="Write final markdown answers from the plan. Be verbose.",
)

user_proxy = autogen.UserProxyAgent(
    name="UserProxy",
    human_input_mode="NEVER",
    code_execution_config=False,
    is_termination_msg=lambda m: "DONE" in m.get("content", ""),
)

Each agent now routes through the same endpoint but resolves to a different provider model. This per-agent routing is the core of mixing GPT-4o and Claude 3.5 Sonnet agents behind one gateway without custom client code.

Step 5: Wire up group chat and run

GroupChat requires a manager LLM to pick speakers. Use GPT-4o for the manager because it handles orchestration well.

group = autogen.GroupChat(
    agents=[user_proxy, planner, drafter],
    messages=[],
    max_round=8,
    allow_repeat=False,
)

manager = autogen.GroupChatManager(
    groupchat=group,
    llm_config=make_llm_config("openai/gpt-4o"),
)

user_proxy.initiate_chat(
    manager,
    message="Plan and draft a 3-step onboarding guide for our API. End with DONE",
)

Set allow_repeat=False so the same agent doesn’t speak twice in a row unless necessary. Run the script; you’ll see Planner output steps, then Drafter expands them.

Step 6: Verify routing and token metering

Check that each message used the intended model. AutoGen stores the response model in the message dict if you patch the agent, but the simplest verification is to log the configured model per agent name:

expected = {"Planner": "openai/gpt-4o", "Drafter": "anthropic/claude-3.5-sonnet"}
for agent in (planner, drafter):
    assert agent.llm_config["model"] == expected[agent.name]
    print(f"{agent.name} -> {agent.llm_config['model']} OK")

For live confirmation, inspect the gateway’s usage metering dashboard: each agent’s tokens appear under the same project, tagged by model slug. Because the gateway provides per-token usage metering, you can attribute cost precisely without client-side accounting.

A successful run terminates when Drafter prints DONE. If you see repeated speaker selections or no termination, tighten is_termination_msg or lower max_round.

Inspecting response headers

If you call the OpenAI client directly in a test, print response.headers.get("x-model"). The gateway echoes the resolved model and flags cache hits. This confirms the cache-control hint from Step 3 was honored.

Step 7: Production hardening

Run group chats with a supervisor that catches exceptions. Wrap initiate_chat in try/except to handle gateway 5xx errors. The endpoint already performs automatic fallback when a provider is rate-limited or degraded, but you should still set a client-side timeout as done in make_llm_config.

For high concurrency, bump max_tokens only where needed; Claude’s tokenizer is efficient for long drafts, GPT-4o for short logic. Keep temperature low on the planner to avoid malformed steps.

Troubleshooting

404 on model slug. Run curl -H "Authorization: Bearer $N4N_API_KEY" $N4N_BASE_URL/models to list valid slugs. Fix the string in make_llm_config.

Double slash in URL. base_url must not end with /. The SDK adds /chat/completions.

Agents ignore termination. Ensure UserProxyAgent has human_input_mode="NEVER" and the termination lambda matches exact casing.

Unexpected token counts. Providers tokenize differently. Compare prompt_tokens per agent, not across models.

Why this pattern holds up

Per-agent model selection exploits model strengths while keeping infrastructure flat. By routing through one OpenAI-compatible endpoint that addresses 240+ models, you avoid SDK sprawl and get uniform logs. The setup to mix GPT-4o and Claude 3.5 Sonnet agents behind one gateway is a template: add a Critic on google/gemini-1.5-pro or a Coder on openai/gpt-4o-mini by calling make_llm_config with a new slug. No new clients, no new env vars.

Keep transcripts. Multi-agent bugs surface only when you read the full group.messages dump. Set max_round conservatively in production and alert on chat loops.