Unit testing CrewAI agent tools is the difference between a multi-agent system that silently corrupts data and one you can refactor with confidence. Most teams wire tools into agents and only find breaks at integration time; that’s backwards. This guide shows how to isolate and test tool logic directly with pytest, mocks, and schema assertions so your CrewAI workflows fail loud in CI, not in production.
Step 1: Scaffold a testable project layout
Start with a standard Python package and a separate tests/ directory. Keep tool definitions in a module that does not import your agent or crew, so tests can load the tool without booting LangChain or kicking off LLM calls.
mkdir -p myproject/tools tests
touch myproject/__init__.py myproject/tools/__init__.py
python -m venv .venv && source .venv/bin/activate
pip install crewai-tools pytest requestsA minimal pyproject.toml with pytest configured avoids path headaches:
[tool.pytest.ini_options]
pythonpath = ["."]
testpaths = ["tests"]Step 2: Write the tool with injected dependencies
The biggest mistake in unit testing CrewAI agent tools is hardcoding requests.get or an SDK client inside _run. Wrap external I/O behind an object you can swap. Below is a WeatherTool built on crewai_tools.BaseTool (a Pydantic model, so custom fields are fine).
# myproject/tools/weather.py
from crewai_tools import BaseTool
from pydantic import BaseModel
class WeatherInput(BaseModel):
city: str
class WeatherClient:
def get(self, city: str) -> str:
# real implementation calls an API
raise NotImplementedError
class WeatherTool(BaseTool):
name: str = "weather_lookup"
args_schema: type[BaseModel] = WeatherInput
client: WeatherClient
def _run(self, city: str) -> str:
return self.client.get(city)For simpler tools, the @tool decorator works, but you lose the explicit dependency slot:
from crewai_tools import tool
@tool("greet")
def greet(name: str) -> str:
"""Greet a user by name."""
return f"Hello, {name}"We’ll focus on the BaseTool pattern because it forces separation of side effects.
Step 3: Test the core execution path
Write a test that constructs the tool with a fake client. This validates business logic without network access.
# tests/test_weather_tool.py
from myproject.tools.weather import WeatherTool, WeatherClient
class FakeClient(WeatherClient):
def get(self, city: str) -> str:
return f"sunny in {city}"
def test_weather_tool_returns_formatted_string():
tool = WeatherTool(client=FakeClient())
result = tool._run("Berlin")
assert result == "sunny in Berlin"Run it:
pytest tests/test_weather_tool.py -qA green check confirms the tool’s internal contract holds. If you later change the formatting, this test catches it before the agent ever sees the output.
Step 4: Validate input schema enforcement
CrewAI uses the args_schema to coerce and validate agent-supplied arguments. Your unit testing CrewAI agent tools suite must confirm that invalid input is rejected, not silently mangled.
import pytest
from pydantic import ValidationError
def test_weather_tool_rejects_non_string_city():
tool = WeatherTool(client=FakeClient())
with pytest.raises(ValidationError):
# CrewAI calls _run after parsing; simulate parse failure
tool.args_schema(city=123)Also test that the tool’s run method (the public wrapper) correctly forwards to _run:
def test_weather_tool_run_wrapper():
tool = WeatherTool(client=FakeClient())
# run() accepts a dict matching the schema
assert tool.run({"city": "Paris"}) == "sunny in Paris"Step 5: Mock external HTTP and LLM calls
Real tools call APIs. Patch at the boundary. If your tool talks to an inference gateway such as n4n.ai’s OpenAI-compatible endpoint, mock the client class so tests don’t incur token usage or hit fallback logic. For a plain requests based client:
# myproject/tools/http_weather.py
import requests
from crewai_tools import BaseTool
from pydantic import BaseModel
class HTTPWeatherInput(BaseModel):
city: str
class HTTPWeatherTool(BaseTool):
name: str = "http_weather"
args_schema: type[BaseModel] = HTTPWeatherInput
def _run(self, city: str) -> str:
resp = requests.get(f"https://api.example.com/weather", params={"q": city}, timeout=5)
resp.raise_for_status()
return resp.json()["condition"]Test with unittest.mock:
from unittest.mock import patch
from myproject.tools.http_weather import HTTPWeatherTool
def test_http_weather_mocked():
fake_json = {"condition": "rainy"}
with patch("myproject.tools.http_weather.requests.get") as mock_get:
mock_get.return_value.status_code = 200
mock_get.return_value.json.return_value = fake_json
tool = HTTPWeatherTool()
assert tool._run("London") == "rainy"
mock_get.assert_called_once_with(
"https://api.example.com/weather",
params={"q": "London"},
timeout=5
)This proves the request shape and response parsing are correct. No network required.
Step 6: Test failure modes and retries
Agents degrade badly when tools throw unstructured exceptions. Unit testing CrewAI agent tools should cover timeouts, non-200s, and empty payloads.
def test_http_weather_handles_500():
with patch("myproject.tools.http_weather.requests.get") as mock_get:
mock_get.return_value.status_code = 500
mock_get.return_value.raise_for_status.side_effect = requests.HTTPError("boom")
tool = HTTPWeatherTool()
with pytest.raises(requests.HTTPError):
tool._run("Tokyo")If you implement retry logic, test the retry count explicitly:
def test_http_weather_retries_three_times(monkeypatch):
calls = {"n": 0}
def flaky_get(*args, **kwargs):
calls["n"] += 1
raise requests.Timeout
monkeypatch.setattr("myproject.tools.http_weather.requests.get", flaky_get)
tool = HTTPWeatherTool()
with pytest.raises(requests.Timeout):
tool._run("Rome")
assert calls["n"] == 3 # assuming you coded max_retries=3Step 7: Test the decorated @tool variant
If your team prefers the lightweight decorator, still assert the wrapped function and the metadata the agent relies on.
from myproject.tools.simple import greet
def test_greet_tool_metadata():
assert greet.name == "greet"
assert "Greet a user" in greet.description
def test_greet_tool_run():
assert greet.run({"name": "Ada"}) == "Hello, Ada"Because greet.func is the original Python function, you can also test it directly for speed.
Step 8: Wire tests into CI and verify success
Add a GitHub Actions step or pre-commit hook that runs pytest. A successful run shows zero network calls (use --disable-socket via pytest-socket if you want strict hermeticity) and all assertions green.
# .github/workflows/test.yml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install -e . pytest pytest-socket
- run: pytest --disable-socketVerification checklist:
pytestexits 0 with no skipped tests.- Coverage on
tools/exceeds 90% (usepytest-cov). - No
requestsor SDK calls escape to real hosts (socket disabled).
Why this discipline pays off
When you treat unit testing CrewAI agent tools as first-class, you decouple agent prompt engineering from tool reliability. A broken schema or a missed HTTP status code surfaces in milliseconds, not after a 20-step agent trace. The patterns above—dependency injection, schema tests, boundary mocking, and failure injection—are the same ones we use for tools that bridge to databases, internal APIs, and LLM gateways. Do them once per tool and your multi-agent system stays debuggable as it grows.