AutoGen config_list basics for n4n.ai's 240+ models

AutoGen’s config_list is the mechanism that lets an agent swap between LLM endpoints without changing application code. When you target the autogen config_list n4n.ai models catalog, you point at a single OpenAI-compatible gateway that fronts 240+ models and handles provider fallback upstream. This guide gives you a concrete, ordered path to stand up that config_list, wire it into agents, and avoid the mistakes that waste tokens in production.

1. What config_list is and isn’t

config_list is a Python list of dictionaries. Each dictionary is one model configuration. AutoGen’s OpenAIWrapper iterates over the list, trying entries until a request succeeds or the list exhausts.

config_list = [
    {
        "model": "openai/gpt-4o",
        "api_key": "sk-your-key",
        "base_url": "https://api.n4n.ai/v1",
        "api_type": "openai",
    }
]

It is not a load balancer. By default AutoGen round-robins across entries for successive calls; it does not weight by latency or cost. If you need strict priority fallback, you must code it with config_list_from_json modes or wrap the wrapper yourself. The list also does not validate model names at construction time—bad strings fail only at request time.

2. Build a minimal config_list for n4n.ai

The gateway exposes one endpoint. You do not need per-provider base URLs. Model names follow the provider/model convention that n4n.ai normalizes. When building the autogen config_list n4n.ai models mapping, prefer provider-prefixed strings exactly as the catalog lists them.

import os

config_list = [
    {
        "model": "openai/gpt-4o-mini",
        "api_key": os.environ["N4N_API_KEY"],
        "base_url": "https://api.n4n.ai/v1",
        "api_type": "openai",
        "temperature": 0.2,
    },
    {
        "model": "anthropic/claude-3-5-sonnet",
        "api_key": os.environ["N4N_API_KEY"],
        "base_url": "https://api.n4n.ai/v1",
        "api_type": "openai",
        "temperature": 0.2,
    },
]

The api_type must be "openai" because the gateway speaks the OpenAI chat completions protocol. Using "azure" or "anthropic" directly will break the request path.

A common pattern is to load this from a JSON file so ops can rotate keys without code changes:

[
  {
    "model": "openai/gpt-4o",
    "api_key": "sk-...",
    "base_url": "https://api.n4n.ai/v1",
    "api_type": "openai"
  }
]

# load with autogen's helper
python -c "from autogen import config_list_from_json; print(config_list_from_json('config.json'))"

If you manage more than a handful of models, generate the file from the catalog API instead of hand-writing it. The autogen config_list n4n.ai models set changes as providers add versions; a script that pulls /v1/models and emits JSON keeps your config honest.

3. Use fallback without double-falling

n4n.ai already performs automatic fallback when a backing provider is rate-limited or degraded. If you also list five models in config_list, AutoGen will retry on a different model after a gateway error. That can be useful for capability differences (e.g., JSON mode missing on one model) but dangerous for cost: a cheap call can silently escalate to a premium model.

Define intent explicitly. If you want gateway-level resilience only, use a single entry and let n4n.ai route:

config_list = [{
    "model": "openai/gpt-4o-mini",
    "api_key": os.environ["N4N_API_KEY"],
    "base_url": "https://api.n4n.ai/v1",
    "api_type": "openai",
}]

If you need client-side capability fallback, isolate it to a separate agent with a narrow list:

reasoning_config = [
    {"model": "anthropic/claude-3-5-sonnet", "base_url": "https://api.n4n.ai/v1", "api_key": os.environ["N4N_API_KEY"], "api_type": "openai"},
    {"model": "openai/gpt-4o", "base_url": "https://api.n4n.ai/v1", "api_key": os.environ["N4N_API_KEY"], "api_type": "openai"},
]

Tradeoff: client-side fallback gives you control over which model substitutes, but you now own the latency and cost matrix. Gateway fallback is invisible and preserves your selected model class.

4. Forward cache-control and routing hints

The gateway honors client routing directives and forwards provider cache-control hints. In AutoGen, pass them via extra_body or headers in the config dict. For Anthropic-backed models, prefix caching is requested with cache_control markers in the message payload; n4n.ai forwards those to the provider.

config_list = [{
    "model": "anthropic/claude-3-5-sonnet",
    "api_key": os.environ["N4N_API_KEY"],
    "base_url": "https://api.n4n.ai/v1",
    "api_type": "openai",
    "headers": {"x-n4n-route": "us-east"},
    "extra_body": {"cache_control": {"type": "ephemeral"}},
}]

Don’t assume the hint is honored by every model. OpenAI models ignore Anthropic-style cache_control; the gateway passes it through but the provider drops it. Test with a small prompt and inspect the usage field for cache hits before relying on it for cost control.

5. Read per-token metering

n4n.ai returns standard OpenAI usage objects. AutoGen exposes them on the underlying client response. After a chat completion, grab response.usage:

from autogen import AssistantAgent, UserProxyAgent, initiate_chat

assistant = AssistantAgent("assistant", llm_config={"config_list": config_list})
proxy = UserProxyAgent("proxy", human_input_mode="NEVER")
result = initiate_chat(assistant, proxy, message="Summarize: ...")
# usage is available on the last client response
print(assistant.client.wrapper.last_response.usage)

If you batch multiple agents, aggregate usage yourself. The gateway meters per token; your side must correlate across agent turns if you need per-task cost. AutoGen’s internal token counting is approximate and should not be used for billing reconciliation.

6. Common pitfalls

api_type mismatch. Engineers copy Azure snippets and leave "api_type": "azure". The gateway is OpenAI-compatible; Azure auth and URL shaping will 401.

Model name drift. n4n.ai uses provider/model. Writing "gpt-4o" without the openai/ prefix fails resolution. Pull the exact string from the model catalog.

Round-robin surprise. With two entries, AutoGen alternates. A latency-sensitive path may hit the slower model every other call. Use config_list_from_json(env_or_file, filter_dict) to pin a single model per deployment.

Secret in source. Hardcoding api_key in the dict is how keys leak. Use env vars or a secrets manager.

Ignoring timeout. Default OpenAI client timeout is 600s. Set timeout in the config to fail fast:

{"model": "openai/gpt-4o", "base_url": "https://api.n4n.ai/v1", "api_key": "...", "api_type": "openai", "timeout": 30}

Temperature per entry. A temperature set in one dict does not apply to others. If you copy a list and forget to adjust, you may get 0.9 randomness on a code-gen model.

7. Tradeoffs: one model vs many

A single-entry config_list is easiest to reason about. You get gateway fallback, one bill, one latency profile. The autogen config_list n4n.ai models approach shines when you genuinely need heterogeneous capabilities—say, a vision model for one step, a cheap classifier for another.

Multiple entries add surface area: more prompt tuning per model, more cost variance, and debugging across provider quirks. Start with one model. Add a second only when a benchmark proves the task needs it. If you later scale to 240+ models, generate the config_list programmatically from the catalog API rather than hand-writing JSON. That keeps the autogen config_list n4n.ai models mapping honest as providers rotate.

8. Quick start checklist

1. Set base_url to the gateway, api_type to openai.
2. Prefix model names with provider.
3. Load keys from env.
4. Start with one model; verify usage metering.
5. Add fallback entries only for capability gaps.
6. Pass cache hints via extra_body and test.

That’s the baseline. From here, wire config_list into GroupChat or Sequential agents and watch the gateway’s fallback absorb provider blips without code changes.