Routing Gemini 3 agent traffic through n4n.ai

Building agents on Gemini 3 means juggling multimodal inputs, native tool use, and the inevitable provider rate limit. gemini 3 n4n.ai routing collapses that complexity into one OpenAI-compatible call path: you send standard chat completion requests, and the gateway handles fallback, cache hints, and per-token metering behind the scenes.

Step 1: Point your client at the gateway for gemini 3 n4n.ai routing

Get an API key from your gateway account and export it as N4N_API_KEY. The only client change is the base_url. Everything else stays OpenAI-shaped, so existing agent code that imports the openai package keeps working.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.n4n.ai/v1",
    api_key=os.environ["N4N_API_KEY"],
)

If you run inside a container, mount the key as a secret rather than baking it into the image. The client instance is thread-safe; construct it once at process startup and share it across worker threads. Don’t bother writing your own exponential backoff for 429s—the routing layer already shifts traffic when a provider returns 429 or 503, and a local retry would just amplify load.

Step 2: Set the model string and routing preferences

Gemini 3 is addressed by its full model ID. The string itself is the primary routing directive. Use google/gemini-3-pro for the flagship multimodal build, or google/gemini-3-flash when you need lower latency on simple turns.

MODEL = "google/gemini-3-pro"

# optional: pass a routing hint via extra headers if you must constrain providers
resp = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "health check"}],
    extra_headers={"X-Route-Prefer": "google"},  # honored by the gateway
)

Keep the hint minimal. Over-specifying routes defeats the automatic fallback that makes this setup worthwhile. In practice, only pin a provider when you have a compliance reason; otherwise let the gateway pick the healthiest endpoint.

Step 3: Send a multimodal agent turn with tools

Agents need to see and act. Gemini 3 accepts interleaved text and image blocks in the OpenAI message format. Define your tools as JSON schemas and let the model emit calls. Use URLs for remote images; base64 works but inflates request size and burns token metering on overhead.

tools = [{
    "type": "function",
    "function": {
        "name": "fetch_invoice",
        "description": "Retrieve invoice PDF by ID",
        "parameters": {
            "type": "object",
            "properties": {"invoice_id": {"type": "string"}},
            "required": ["invoice_id"],
        },
    },
}]

messages = [
    {"role": "system", "content": "You are a document agent."},
    {"role": "user", "content": [
        {"type": "text", "text": "Extract line items from this scan."},
        {"type": "image_url", "image_url": {"url": "https://example.com/scan.jpg"}},
    ]},
]

resp = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

The response mirrors OpenAI’s structure: resp.choices[0].message.tool_calls holds the model’s intended actions. If the model decides no tool is needed, you get content directly. Gemini 3 can emit parallel tool calls in one message; your dispatcher must handle a list, not a single call.

Step 4: Run the tool-call loop

A real agent loops until the model stops requesting tools. Write the loop once; the routing is irrelevant inside it. Keep tool execution isolated so a failure in one call doesn’t crash the turn.

import json

def run_agent(client, model, messages, tools):
    while True:
        resp = client.chat.completions.create(
            model=model, messages=messages, tools=tools)
        msg = resp.choices[0].message
        messages.append(msg)
        if not msg.tool_calls:
            return msg.content
        for call in msg.tool_calls:
            args = json.loads(call.function.arguments)
            try:
                result = dispatch(call.function.name, args)
            except Exception as e:
                result = f"error: {e}"
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": result,
            })

dispatch is your local function registry. Keep it side-effect free where possible; Gemini 3 may call the same tool multiple times across turns, and idempotency saves you from double charges or duplicate state changes.

Step 5: Handle streaming and cache hints

For interactive agents, stream tokens. Pass stream=True and iterate. The gateway forwards provider cache-control hints, so if your underlying provider supports prefix caching, mark static system prompts accordingly in the message payload.

stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "system", "content": "Long static instructions..."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

When streaming with tools, the tool-call deltas arrive incrementally; buffer them and reconstruct the full call before dispatching. Don’t implement client-side fallback for 429s. The routing layer detects degraded providers and reroutes to a healthy one without changing your code. That’s the entire point of fronting Gemini 3 with a gateway.

Step 6: Verify success

Verification is two-fold: confirm the model responded and confirm usage metering is present. Set BASE_URL to your gateway’s OpenAI-compatible URL (the same one from Step 1) and run a minimal curl:

curl -s $BASE_URL/chat/completions \
  -H "Authorization: Bearer $N4N_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"google/gemini-3-pro","messages":[{"role":"user","content":"say pong"}]}' \
  | jq '.model, .usage'

You should see "google/gemini-3-pro" (or the resolved fallback ID) and a non-null usage object with prompt_tokens and completion_tokens. In Python, assert on resp.model and resp.usage.total_tokens > 0.

For agent flows, add a unit test that mocks dispatch and checks the loop terminates with a final assistant message containing no tool_calls. If the gateway fails over to a backup provider, the response model field may differ from your request—handle that gracefully in logs rather than asserting strict equality.

Operational notes

Treat the gateway as a thin, smart proxy. You still own prompt design, tool schemas, and error handling for business logic. What you offload is provider heterogeneity, rate-limit roulette, and cache plumbing.

When you scale Gemini 3 agent traffic, batch independent turns and reuse the client instance to avoid TLS handshake overhead. Spawn as many concurrent requests as your local CPU allows; the gateway handles the fan-out to providers.

If you need to pin a specific provider for compliance, use the routing header shown in Step 2, but accept the trade-off: you lose automatic fallback. For most agent workloads, letting the gateway choose is the right default.

That’s the whole integration. No custom retry library, no provider SDK forks—just OpenAI-compatible calls with a different base URL and a model string that targets Gemini 3.