Configuring CrewAI's LLM class for n4n.ai endpoints

Getting the crewai llm class n4n.ai endpoint config right is the difference between a crew that routes through a single OpenAI-compatible gateway and one that silently bypasses it and hits vendor APIs directly. Below is the exact sequence we use to point CrewAI at the gateway, run a trivial crew, and confirm tokens are metered at the edge.

Step 1: Install CrewAI in an isolated environment

CrewAI pulls in langchain, litellm, and a few heavy transitive deps. Pin versions so the LLM class interface does not shift under you.

python -m venv .venv
source .venv/bin/activate
pip install "crewai==0.74.0" "python-dotenv==1.0.1"

The LLM class we import later is available in CrewAI >= 0.30. If from crewai import LLM raises an ImportError, upgrade before proceeding.

Step 2: Store the gateway API key

The gateway uses one bearer token for all 240+ models. Put it in .env so it never lands in source control.

echo "N4N_API_KEY=sk-your-key-here" >> .env

Load it explicitly in your entrypoint:

from dotenv import load_dotenv
import os

load_dotenv()
API_KEY = os.getenv("N4N_API_KEY")
assert API_KEY, "N4N_API_KEY not found in environment"

Do not hardcode the key in the LLM constructor inside shared modules. Read it from env at runtime.

You can sanity-check connectivity before writing any CrewAI code:

curl https://api.n4n.ai/v1/models \
  -H "Authorization: Bearer $N4N_API_KEY" | head -c 300

An OpenAI-style JSON list of model objects confirms the credential and endpoint are live.

Step 3: Instantiate the LLM class with the gateway base URL

CrewAI’s LLM is a thin wrapper over LiteLLM. To target the gateway, set base_url to the OpenAI-compatible endpoint and prefix the model with openai/ so LiteLLM emits the correct request shape.

from crewai import LLM

llm = LLM(
    model="openai/gpt-4o-mini",
    api_key=API_KEY,
    base_url="https://api.n4n.ai/v1",
    temperature=0.2,
    timeout=30,
    max_retries=2,
)

If you omit the openai/ prefix, LiteLLM attempts to resolve the model against its own provider registry and will ignore base_url, calling api.openai.com instead. The prefix forces the OpenAI-compatible path.

For the crewai llm class n4n.ai endpoint config, the three non-negotiable fields are model, api_key, and base_url. Everything else is latency or output-shaping tuning.

Passing cache hints

The gateway honors provider cache-control hints. Forward them via extra_headers:

llm_cached = LLM(
    model="openai/gpt-4o-mini",
    api_key=API_KEY,
    base_url="https://api.n4n.ai/v1",
    extra_headers={"x-cache-control": "ephemeral"},
)

This is optional but cuts tail latency on repeated system prompts.

Step 4: Attach the LLM to agents

Agents take an llm argument. Define a minimal agent that uses the configured instance and enables verbose logging so we can see token counts.

from crewai import Agent, Task, Crew

researcher = Agent(
    role="Echo",
    goal="Repeat the user's prompt verbatim",
    backstory="A terse bot that confirms connectivity.",
    llm=llm,
    verbose=True,
)

task = Task(
    description="Say: pong",
    expected_output="The word pong",
    agent=researcher,
)

Do not set api_key on the agent. CrewAI inherits credentials from the LLM object. If you need a second model in the same crew, construct a second LLM with a different model string but the same base_url:

llm_large = LLM(
    model="openai/llama-3-70b",
    api_key=API_KEY,
    base_url="https://api.n4n.ai/v1",
    temperature=0.0,
)

Both agents still route through the same gateway; only the upstream model changes.

Step 5: Run a minimal crew and capture output

Execute the crew and print the result. With verbose=True, CrewAI logs LiteLLM token usage.

crew = Crew(agents=[researcher], tasks=[task])
result = crew.kickoff()
print("RESULT:", result)

A successful run prints RESULT: pong and, somewhere in the agent logs, a line similar to:

[INFO] Token usage: prompt=12 completion=1 total=13

If you instead see connection errors to api.openai.com, the base_url was ignored—almost always because the model string lacked the openai/ prefix. Fix the prefix, not the network.

Step 6: Confirm routing and fallback behavior

Check the gateway usage panel or response headers for the resolved provider. Because the gateway provides automatic fallback when a provider is rate-limited or degraded, a single base_url keeps your crew running even if the primary backend throttles. You do not need try/except blocks for 429s at the CrewAI layer; the gateway shifts traffic and returns a normal completion.

To prove your traffic is hitting the gateway and not a vendor, send a request with a deliberately unknown model ID:

bad_llm = LLM(
    model="openai/does-not-exist-123",
    api_key=API_KEY,
    base_url="https://api.n4n.ai/v1",
)
try:
    Crew(
        agents=[Agent(role="x", goal="x", backstory="x", llm=bad_llm)],
        tasks=[Task(description="test", expected_output="test")],
    ).kickoff()
except Exception as e:
    print("Gateway rejected:", e)

A gateway-formatted error (structured JSON, not a DNS failure) confirms the endpoint config is live.

Step 7: Verify with a direct curl round-trip

Before shipping, mirror the CrewAI call with raw HTTP to remove framework variables from the equation:

curl https://api.n4n.ai/v1/chat/completions \
  -H "Authorization: Bearer $N4N_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "pong"}],
    "temperature": 0.2
  }'

If this returns a completion and the CrewAI run above also returned one, the full stack is aligned.

Verification checklist

• .env loaded, API_KEY non-empty at runtime.
• LLM constructed with base_url="https://api.n4n.ai/v1" and model="openai/...".
• Agent verbose=True shows token counts in logs.
• Network trace shows no DNS lookups to api.openai.com.
• Gateway usage panel shows the request with model name and token totals.
• Unknown-model test yields a gateway error, not a connection timeout.

If all six hold, the crewai LLM class endpoint config is correct and your crews are routing through the gateway as intended.

Common pitfalls

Missing provider prefix. model="gpt-4o" without openai/ makes LiteLLM call OpenAI directly. Always prefix.

Trailing slash on base URL. Some gateways reject https://api.n4n.ai/v1/. Use no trailing slash.

Temperature out of range. LiteLLM passes it through; values >2.0 are rejected upstream. Keep 0–1.

Key committed to git. Use .env and a .gitignore entry. Rotate the key if it leaks.

Retries double-counted. Setting max_retries high on both LiteLLM and a custom HTTP client can cause duplicate spends. Default of 2 is sane.

Following these steps gives you a reproducible, gateway-backed CrewAI setup without writing custom middleware or vendor-specific adapters.