Configure OAI_CONFIG_LIST for AutoGen with n4n.ai

Setting up autogen oai_config_list n4n.ai lets you point Microsoft AutoGen agents at a single OpenAI-compatible gateway that fronts more than 240 models with automatic fallback when a provider is degraded. This walkthrough builds a working config from scratch and validates it with a minimal agent script so you can trust it in production.

Step 1: Install AutoGen in a clean environment

Create a virtual environment and install the classic pyautogen package, which still ships the config_list_from_json helper and the ConversableAgent abstractions most teams use.

python -m venv .venv
source .venv/bin/activate
pip install pyautogen python-dotenv

If you are on AutoGen 0.4+ and using the autogen-agentchat surface, the llm_config dict shape is identical; only the agent constructors differ. The config list format is stable across versions, so the file you write here will outlive a framework upgrade. Pin the version in your requirements file to avoid surprise breaking changes in the wrapper layer.

Step 2: Understand the OAI_CONFIG_LIST contract

AutoGen expects OAI_CONFIG_LIST to be a JSON array of objects. Each object describes one LLM endpoint or one model alias. At minimum you need model and a way to authenticate. For an OpenAI-compatible server you also supply base_url and either api_key or api_key_env_var.

A bare OpenAI entry looks like this:

[
  {
    "model": "gpt-4o-mini",
    "api_key_env_var": "OPENAI_API_KEY",
    "api_type": "openai"
  }
]

When AutoGen initializes a client, it reads api_key_env_var from the process environment, so you never hard-code secrets in the JSON file. The api_type defaults to "openai" but being explicit avoids ambiguity.

AutoGen’s client wrapper filters this list at request time. If you pass filter_dict={"api_type": "openai"} to config_list_from_json, only openai-type entries load. At inference, the agent picks the first matching config; on a retryable error it walks down the list. This client-side rotation complements server-side fallback—you get two independent layers of resilience.

Step 3: Define the autogen oai_config_list n4n.ai entry

Replace the base URL with the single OpenAI-compatible endpoint the gateway exposes and reference your key via an env var. Because the gateway routes to many providers, the model field accepts qualified names such as anthropic/claude-3.5-sonnet or openai/gpt-4o.

Create a file named OAI_CONFIG_LIST.json:

[
  {
    "model": "anthropic/claude-3.5-sonnet",
    "base_url": "https://api.n4n.ai/v1",
    "api_type": "openai",
    "api_key_env_var": "N4N_API_KEY"
  },
  {
    "model": "openai/gpt-4o-mini",
    "base_url": "https://api.n4n.ai/v1",
    "api_type": "openai",
    "api_key_env_var": "N4N_API_KEY"
  }
]

The second entry is not required, but listing two models gives AutoGen its own client-side retry if the first model returns a non-retryable error. The gateway itself already performs automatic fallback when an upstream provider is rate-limited, so you get defense in depth without writing custom retry code.

Export the key before running any script:

export N4N_API_KEY="sk-your-real-key"

Keep this file out of version control. A .gitignore entry for OAI_CONFIG_LIST.json is mandatory; treat it like a docker-compose override that points at secrets.

Step 4: Load the config list in Python

AutoGen provides config_list_from_json to load from a file or environment variable. Point it at your JSON file:

from autogen import config_list_from_json

config_list = config_list_from_json(
    env_or_file="OAI_CONFIG_LIST.json",
    filter_dict={"api_type": "openai"}
)

assert len(config_list) > 0, "config list is empty"
print(f"Loaded {len(config_list)} endpoint configs")

If you prefer environment-variable only deployment, set OAI_CONFIG_LIST to the JSON string and call config_list_from_json(env_or_file="OAI_CONFIG_LIST"). The helper parses either a path or a raw JSON string. In containerized deployments we usually mount the JSON as a secret file and reference the path—this keeps the config inspectable without baking it into the image.

Step 5: Wire the config into an AutoGen agent

The simplest smoke test is a AssistantAgent that answers a trivial question and a UserProxyAgent that terminates after one reply. Disable code execution so the test runs without Docker.

from autogen import ConversableAgent, UserProxyAgent

llm_config = {
    "config_list": config_list,
    "temperature": 0.0,
}

assistant = ConversableAgent(
    name="assistant",
    llm_config=llm_config,
    system_message="You are a terse engineering assistant."
)

user_proxy = UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=1,
    code_execution_config=False,
)

user_proxy.initiate_chat(
    assistant,
    message="Reply with the single word: pong"
)

The config_list is passed straight to the underlying OpenAI client wrapper. Because base_url is set per entry, every request leaves your process bound for the gateway rather than OpenAI directly. If you later swap models, you edit the JSON, not the Python.

Step 6: Verify success and inspect token metering

Run the script:

python smoke_test.py

You should see a chat transcript ending with the assistant message pong. To confirm the gateway handled the request and that per-token metering is flowing, inspect the assistant’s chat history for usage data returned by the API:

for msg in assistant.chat_history:
    if isinstance(msg, dict) and "usage" in msg:
        print("usage:", msg["usage"])

If you see prompt_tokens and completion_tokens greater than zero, the call succeeded end-to-end. The gateway’s per-token usage metering means those counts match what will appear on your invoice—no separate reconciliation needed.

A failure mode to watch: if N4N_API_KEY is unset, the client raises AuthenticationError immediately. If the model string is not routed by the gateway, you get a 404 from the chat completions path. Both are surfaced as clear Python exceptions, not silent fallbacks. When the gateway fails over upstream, your request still returns 200 with valid tokens, so the history check is the only signal that a secondary provider served the traffic.

Step 7: Add client-side routing directives and cache hints

For production fleets you often want to pin a request to a specific provider region or enable prompt caching. The gateway honors client routing directives and forwards provider cache-control hints when sent as request headers. In AutoGen you can inject headers via the extra_headers key in the llm_config (supported by the underlying OpenAI SDK through the wrapper):

llm_config = {
    "config_list": config_list,
    "extra_headers": {
        "x-route-directive": "us-east",
        "cache-control": "max-age=300"
    },
}

This is optional. If you omit it, the gateway applies its default routing and caching policy. Keep in mind that AutoGen may not forward extra_headers on every internal call in older versions; verify against your installed release by checking the outgoing request logs or a local proxy. When it works, the cache hint flows to the upstream provider that supports it, cutting latency and cost on repeated system prompts.

You can also extend the config list with a third entry pointing at a local vLLM instance for air-gapped testing. AutoGen will try the gateway first, then the local model, with zero code changes.

Troubleshooting checklist

Empty config list: config_list_from_json returns [] if the file path is wrong or the JSON is invalid. Run python -m json.tool OAI_CONFIG_LIST.json to validate syntax.
Model not found: The gateway fronts 240+ models, but each model id must match its qualified name. List available models from your gateway dashboard and copy exactly, including the provider prefix.
SSL errors: If you are behind a corporate proxy, set REQUESTS_CA_BUNDLE before launching Python.
Rate limits: The gateway automatically fails over when a provider is rate-limited, but AutoGen’s own retry loop uses max_retries in the client config. Add "max_retries": 3 to each entry if you see transient 429s.
Wrong base_url: Forgetting the /v1 suffix is the most common copy-paste mistake. The OpenAI SDK appends /chat/completions to base_url, so the full path must resolve to …/v1/chat/completions.

Closing notes

You now have a reproducible autogen oai_config_list n4n.ai setup that decouples your agent code from provider-specific SDKs. Because the config is just data, you can ship it as a secret-referenced file in CI and rotate models without touching application logic. The next step is to parameterize model per agent role—planner, coder, critic—and let the gateway handle the routing complexity while AutoGen handles the orchestration.