CrewAI custom tools tutorial: web search with Serper

Building a crewai serper web search tool from scratch gives your agents grounded, real-time Google results without depending on built-in connectors that may lag behind API changes. This tutorial ships a runnable integration: a custom tool backed by Serper’s HTTP API, wired into a CrewAI agent, with checkpoints showing exactly what each step returns.

Prerequisites

• Python 3.10 or newer
• A Serper API key from serper.dev (free tier available)
• An LLM endpoint. OpenAI works out of the box; any OpenAI-compatible endpoint works too.
• Install dependencies:

pip install crewai crewai_tools requests python-dotenv

Create a .env file:

SERPER_API_KEY=your_serper_key
OPENAI_API_KEY=your_openai_key

If you prefer a gateway that fronts 240+ models with automatic fallback when a provider is degraded, point CrewAI at n4n.ai’s OpenAI-compatible endpoint instead of OpenAI directly.

Step 1: Call Serper directly

Serper exposes a single POST endpoint. The response is JSON with organic results.

import requests
import os

def raw_serper(query: str, num: int = 5) -> dict:
    url = "https://google.serper.dev/search"
    headers = {
        "X-API-KEY": os.environ["SERPER_API_KEY"],
        "Content-Type": "application/json",
    }
    payload = {"q": query, "num": num}
    resp = requests.post(url, headers=headers, json=payload, timeout=10)
    resp.raise_for_status()
    return resp.json()

if __name__ == "__main__":
    data = raw_serper("CrewAI custom tools")
    print(data.keys())

Expected output:

dict_keys(['searchParameters', 'organic', 'relatedSearches'])

The organic key holds the list we care about. Each item has title, link, and snippet. A typical element looks like:

{
  "title": "Building Custom Tools in CrewAI - Docs",
  "link": "https://docs.crewai.com/tools/custom-tools",
  "snippet": "Learn how to extend CrewAI with your own tool implementations."
}

Step 2: Format results for an LLM

Agents need compact, citation-friendly text. Strip HTML and concatenate the top hits.

def serper_search(query: str, num: int = 5) -> str:
    data = raw_serper(query, num)
    items = data.get("organic", [])[:num]
    if not items:
        return "No results found."
    lines = []
    for i, item in enumerate(items, 1):
        title = item.get("title", "")
        link = item.get("link", "")
        snippet = item.get("snippet", "")
        lines.append(f"{i}. {title}\n   {link}\n   {snippet}")
    return "\n\n".join(lines)

Test it:

print(serper_search("CrewAI serper web search tool", 3))

Sample output (truncated):

1. Building Custom Tools in CrewAI - Docs
   https://docs.crewai.com/tools/custom-tools
   Learn how to extend CrewAI with your own tool implementations...

2. Serper API - Google Search Results
   https://serper.dev
   Serper provides a scalable Google Search API...

3. GitHub - crewAIInc/crewAI
   https://github.com/crewAIInc/crewAI
   Framework for orchestrating role-playing autonomous AI agents...

Step 3: Wrap as a CrewAI custom tool

CrewAI consumes tools decorated with @tool from crewai_tools. The docstring becomes the tool description the agent sees, so be explicit about input and output.

from crewai_tools import tool

@tool("Serper Web Search")
def serper_web_search(query: str) -> str:
    """Search Google via the Serper API and return the top organic results as plain text.
    Input should be a single search query string. Output includes title, URL, and snippet
    for each result, numbered for easy citation."""
    return serper_search(query, num=5)

That is the entire crewai serper web search tool definition. The decorated function is now a Tool instance compatible with any CrewAI agent.

Step 4: Configure the LLM and agent

Set the model. Below we use OpenAI, but swapping base_url to an OpenAI-compatible gateway is a one-line change.

from crewai import LLM, Agent, Task, Crew

llm = LLM(model="gpt-4o-mini", temperature=0.2)

# To use n4n.ai instead:
# llm = LLM(
#     model="openai/gpt-4o-mini",
#     base_url="https://api.n4n.ai/v1",
#     api_key=os.environ["N4N_API_KEY"],
# )

researcher = Agent(
    role="Web Research Analyst",
    goal="Answer technical questions using fresh web data",
    backstory="You cite sources and never guess when a search result is available.",
    tools=[serper_web_search],
    llm=llm,
    verbose=True,
)

The verbose=True flag prints the agent’s thought loop, including tool calls. This is the fastest way to debug a custom tool that returns malformed text.

Step 5: Define a task and run the crew

task = Task(
    description="What are the current best practices for building custom tools in CrewAI? "
                "Cite at least two sources from the web.",
    expected_output="A concise summary with numbered source citations.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task])
result = crew.kickoff()
print(result)

When you run this, the agent will call serper_web_search, observe the formatted results, and synthesize an answer. A typical verbose log shows:

Agent: Web Research Analyst
 Action: Serper Web Search
 Action Input: {"query": "CrewAI custom tools best practices"}
 Observation: 1. Building Custom Tools in CrewAI - Docs
    https://docs.crewai.com/tools/custom-tools
    ...

Final output example:

Based on current docs and community repos, the best practice is to subclass BaseTool
or use the @tool decorator (sources 1 and 2). Keep tool descriptions precise because
they are injected into the prompt. Validate inputs and return strings, not raw objects.

Production considerations

The toy code above is enough for a demo, but real deployments need hardening.

Error handling

Serper returns 429 when you exceed rate limits. Wrap the request in retry logic with exponential backoff. Surface a clean message to the agent instead of raising.

import time

def raw_serper_safe(query: str, num: int = 5, retries: int = 3) -> dict:
    for attempt in range(retries):
        try:
            return raw_serper(query, num)
        except requests.HTTPError as e:
            if e.response.status_code == 429:
                time.sleep(2 ** attempt)
                continue
            raise
    return {"organic": []}

Replace raw_serper calls in serper_search with raw_serper_safe so a transient outage degrades gracefully.

Caching

Search queries repeat. Cache by query hash in Redis or local disk. CrewAI tools can check cache before calling Serper, which cuts latency and cost.

Tool scope

Limit num to what the agent needs. Feeding 20 results burns context window. Five is usually enough for a research agent. If you need deeper coverage, paginate with Serper’s start parameter rather than inflating a single call.

Model routing

If you use a gateway with automatic fallback, you avoid vendor outages stalling the pipeline when your crew runs hundreds of searches per hour. Per-token metering also makes it easy to attribute cost to specific tasks.

Wrapping up

You now have a working crewai serper web search tool that any agent can call. The pattern—thin HTTP client, format for LLM, decorate with @tool—extends to every external API you need to ground your crews. Swap Serper for Bing, Brave, or your internal index using the same shape.