CrewAI makes it easy to stand up multi-agent pipelines, but the default pattern of pointing every agent at the same model wastes money and caps overall quality. Proper crewai llm per agent role assignment means giving your planner a strong reasoning model, your researcher a long-context model, and your writer a cheap instruction-tuned model. This tutorial builds a three-agent research crew with distinct LLMs wired to each role, using an OpenAI-compatible gateway so you can swap providers without touching agent code.
Prerequisites
- Python 3.10 or newer
- A virtual environment tool (
venvorconda) crewaiandpython-dotenvinstalled- An API key for an OpenAI-compatible endpoint. We’ll use n4n.ai’s single endpoint that fronts 240+ models with automatic fallback and per-token metering, but any compliant gateway works.
- Familiarity with environment variables and basic Python classes
Set up the project:
mkdir crewai-roles && cd crewai-roles
python -m venv .venv && source .venv/bin/activate
pip install crewai python-dotenvCreate a .env file:
N4N_API_KEY=sk-your-key-hereStep 1: Load config and define base constants
Keep credentials out of code. A small config.py module reads the environment and exposes the base URL.
# config.py
import os
from dotenv import load_dotenv
load_dotenv()
API_KEY = os.getenv("N4N_API_KEY")
BASE_URL = "https://api.n4n.ai/v1" # OpenAI-compatible, 240+ modelsThe BASE_URL is the only network detail your agents need. Because the gateway speaks the OpenAI chat protocol, CrewAI’s LLM wrapper treats it like any other OpenAI endpoint.
Step 2: Instantiate role-specific LLMs
CrewAI’s LLM class (built on litellm) accepts a model string, base_url, api_key, and sampling params. Create one instance per role. Do not reuse a single instance with mutated temperature—the object is cached and shared.
# llms.py
from crewai import LLM
from config import API_KEY, BASE_URL
planner_llm = LLM(
model="openai/o3-mini", # strong reasoning, low temp
base_url=BASE_URL,
api_key=API_KEY,
temperature=0.2,
)
researcher_llm = LLM(
model="anthropic/claude-3-5-sonnet", # 200k context, good retrieval
base_url=BASE_URL,
api_key=API_KEY,
temperature=0.3,
)
writer_llm = LLM(
model="meta/llama-3.1-8b-instruct", # cheap, fast drafting
base_url=BASE_URL,
api_key=API_KEY,
temperature=0.7,
)Expected checkpoint: import the module and print the model names.
>>> from llms import planner_llm, researcher_llm, writer_llm
>>> planner_llm.model, researcher_llm.model, writer_llm.model
('openai/o3-mini', 'anthropic/claude-3-5-sonnet', 'meta/llama-3.1-8b-instruct')If you see the three distinct strings, the assignment is wired correctly before any agent runs.
Step 3: Define agents with assigned LLMs
Pass the llm argument to each Agent. Set allow_delegation=False on the writer to prevent it from spawning sub-agents that would inherit the wrong model.
# agents.py
from crewai import Agent
from llms import planner_llm, researcher_llm, writer_llm
planner = Agent(
role="Content Planner",
goal="Outline a comprehensive article on {topic}",
backstory="Senior editor who structures complex technical subjects",
llm=planner_llm,
verbose=True,
allow_delegation=False,
)
researcher = Agent(
role="Researcher",
goal="Find authoritative sources and extract facts on {topic}",
backstory="Analyst with meticulous citation habits",
llm=researcher_llm,
verbose=True,
allow_delegation=False,
)
writer = Agent(
role="Writer",
goal="Draft the final article from the plan and research",
backstory="Clear, concise tech writer",
llm=writer_llm,
verbose=True,
allow_delegation=False,
)Step 4: Tasks and crew wiring
Tasks declare context dependencies so CrewAI passes prior outputs forward. The researcher needs the planner’s outline; the writer needs both.
# crew.py
from crewai import Task, Crew, Process
from agents import planner, researcher, writer
plan_task = Task(
description="Create a detailed outline for an article on {topic}",
expected_output="Markdown outline with H2/H3 sections",
agent=planner,
)
research_task = Task(
description="Gather key facts and sources for the outline",
expected_output="Bullet list of facts with URLs",
agent=researcher,
context=[plan_task],
)
write_task = Task(
description="Write the article using plan and research",
expected_output="Final markdown article",
agent=writer,
context=[plan_task, research_task],
)
crew = Crew(
agents=[planner, researcher, writer],
tasks=[plan_task, research_task, write_task],
process=Process.sequential,
)Step 5: Run and verify routing
Execute the crew with a concrete topic:
# run.py
from crew import crew
result = crew.kickoff(inputs={"topic": "edge inference for LLMs"})
print(result)Verbose logs should show each agent hitting its own model. A representative truncated log:
[Content Planner] Using model openai/o3-mini
# Edge Inference for LLMs
## Why it matters
### Cost at scale
...
[Researcher] Using model anthropic/claude-3-5-sonnet
- Fact: llama.cpp supports 4-bit quant (source: github.com/ggerganov/llama.cpp)
- Fact: Edge TPUs deliver 4 TOPS/W (source: coral.ai)
[Writer] Using model meta/llama-3.1-8b-instruct
Final draft: "Edge inference moves model execution to the device..."If the planner’s outline appears before the researcher’s bullets, and the writer’s text references those bullets, the context chain and per-role LLMs are both working.
Step 6: Why crewai llm per agent role assignment pays off
Capability matching
A reasoning model like o3-mini excels at structuring an outline but is overkill for turning bullets into prose. A small llama-3.1-8b model drafts acceptably and costs a fraction of the tokens. Researcher roles benefit from large context windows that cheaper models lack.
Cost control
Metering per token at the gateway shows the split. With n4n.ai, per-token usage metering attributes spend to each role’s model call, so you can see the writer burned 2k tokens while the planner used 800.
Latency
Cheap models return faster. Parallelizing independent tasks with Process.hierarchical works better when low-latency roles aren’t blocked on a heavy model.
Step 7: Common pitfalls
Shared LLM object. If you write llm = LLM(...) and pass it to three agents, then later mutate llm.temperature, all agents see the change. Always construct separate instances.
Wrong model string. litellm expects provider prefixes (openai/, anthropic/, meta/). An unprefixed gpt-4o may route to a default provider and ignore your base_url.
Verbose noise. Set verbose=False in production, but keep it on during development to confirm crewai llm per agent role assignment is honored.
Step 8: Dynamic selection (optional)
For variable workloads, build a factory:
def llm_for(role: str, complexity: int) -> LLM:
if role == "planner":
model = "openai/o3-mini" if complexity > 5 else "openai/gpt-4o-mini"
elif role == "researcher":
model = "anthropic/claude-3-5-sonnet"
else:
model = "meta/llama-3.1-8b-instruct"
return LLM(model=model, base_url=BASE_URL, api_key=API_KEY)Pass the returned object into the agent at construction. This keeps the same per-role assignment pattern while adapting to runtime signals.
Recap
You defined three LLMs, attached each to a distinct CrewAI agent, and ran a sequential crew that respects those bindings. The gateway we used forwards provider cache-control hints and honors client routing directives, so each role’s caching strategy stays intact without extra code. With this structure, you can swap any model, add fallback, or introduce a new role without rewiring the whole pipeline.