Building a tool-calling agent in Haystack 2.0

This haystack 2.0 tool-calling agent tutorial shows how to build an agent that invokes external functions using Haystack 2.0’s Agent component and a standard OpenAI-compatible chat model. We’ll define typed tools, wire them into a pipeline, and verify the call loop end to end. No abstractions hidden; you’ll see the exact objects passed between components.

Step 1: Install Haystack 2.0 and set up a clean environment

Create a virtual environment and install the framework plus the OpenAI client. Haystack 2.0 requires Python 3.8+; I use 3.11.

python -m venv .venv
source .venv/bin/activate
pip install haystack-ai openai python-dotenv

Store credentials in a .env file. The OpenAIChatGenerator reads OPENAI_API_KEY by default, but we’ll pass it explicitly to avoid ambient environment surprises.

import os
from dotenv import load_dotenv

load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

If you skip the key, the generator raises at instantiation. That failure is cheap and early—good.

Step 2: Define tools as Haystack Tool objects

Haystack expects tools as instances of Tool with a name, description, JSON schema for parameters, and a callable. The schema must be a valid JSON Schema object. The agent sends this schema to the model; the model returns arguments that Haystack validates before calling your function.

from haystack.tools import Tool

def get_weather(city: str) -> str:
    # Mock implementation; swap for a real API call.
    return f"Sunny in {city}, 22°C"

weather_tool = Tool(
    name="get_weather",
    description="Returns current weather for a given city.",
    parameters={
        "type": "object",
        "properties": {
            "city": {"type": "string", "description": "City name, e.g. Berlin"}
        },
        "required": ["city"],
    },
    function=get_weather,
)

def multiply(a: float, b: float) -> str:
    return str(a * b)

calc_tool = Tool(
    name="multiply",
    description="Multiplies two numbers and returns the product as string.",
    parameters={
        "type": "object",
        "properties": {
            "a": {"type": "number"},
            "b": {"type": "number"},
        },
        "required": ["a", "b"],
    },
    function=multiply,
)

tools = [weather_tool, calc_tool]

Keep tool functions side-effect free in tests. The agent will call them repeatedly if the model loops, so idempotency matters.

Step 3: Configure a chat generator that supports tool calls

The Agent needs a chat generator compatible with native function calling. gpt-4o-mini is cheap and reliable for this. Point the generator at your provider.

from haystack.components.generators.chat import OpenAIChatGenerator

chat_generator = OpenAIChatGenerator(
    model="gpt-4o-mini",
    api_key=OPENAI_API_KEY,
    generation_kwargs={"temperature": 0.0},
)

If you want automatic fallback when a provider is rate-limited or degraded, point the same generator at an OpenAI-compatible gateway such as n4n.ai, which forwards provider cache-control hints and meters per-token usage without changing your Haystack code—just set api_base_url and api_key.

chat_generator = OpenAIChatGenerator(
    model="openai/gpt-4o-mini",
    api_base_url="https://api.n4n.ai/v1",
    api_key=os.getenv("N4N_API_KEY"),
    generation_kwargs={"temperature": 0.0},
)

Either path works; the rest of this haystack 2.0 tool-calling agent tutorial is identical.

Step 4: Instantiate the Agent component

The Agent wraps the generator and tools, manages the reasoning loop, and emits messages. Set max_iterations to bound runaway loops. A system prompt focuses the model on using tools instead of guessing.

from haystack.components.agents import Agent

agent = Agent(
    chat_generator=chat_generator,
    tools=tools,
    system_prompt="You are a precise assistant. Use the provided tools for any factual or computational query.",
    max_iterations=5,
)

The agent is a standard Haystack component: it exposes run() with prompt or messages and returns {"messages": [...], "answers": [...]}.

Step 5: Wrap the agent in a Pipeline

Haystack 2.0 pipelines are directed graphs of components. For a single-agent setup the graph is trivial, but putting the agent in a pipeline keeps the door open for preprocessors or post-processors later.

from haystack import Pipeline

pipe = Pipeline()
pipe.add_component("agent", agent)

result = pipe.run(
    {"agent": {"prompt": "What is 12 times 8 and the weather in Lisbon?"}}
)

The pipeline input dict keys must match component names and their run parameters. Here agent expects prompt. If you pass messages instead, use {"agent": {"messages": [...]}}.

Step 6: Run the agent and verify tool execution

Execute the script. The agent should call multiply and get_weather, then synthesize a final answer. Inspect the message trail to confirm the tools fired.

messages = result["agent"]["messages"]
for m in messages:
    print(f"{m.role}: {m.content}")

# Verify at least one tool call happened
tool_calls = [m for m in messages if m.role == "tool"]
assert len(tool_calls) >= 2, "Expected both tools to be called"
print("SUCCESS: agent invoked", len(tool_calls), "tools")

A successful run prints assistant text, tool results, and a final assistant summary. The assertion proves the loop executed external functions rather than hallucinating numbers. That is the core verification step in this haystack 2.0 tool-calling agent tutorial.

Example output snippet:

assistant: I'll check the weather and do the math.
tool: Sunny in Lisbon, 22°C
tool: 96.0
assistant: 12 times 8 is 96. The weather in Lisbon is sunny at 22°C.
SUCCESS: agent invoked 2 tools

Step 7: Handle failures and observe the loop

Tool functions throw in production. By default Haystack surfaces the exception as a tool message, and the agent can recover if the model retries with corrected arguments. To avoid hard crashes during development, wrap tool bodies and return error strings.

def get_weather(city: str) -> str:
    try:
        # real call here
        return f"Sunny in {city}, 22°C"
    except Exception as e:
        return f"ERROR: {e}"

Set raise_on_failure=False on the agent if you want the pipeline to return partial results instead of propagating. Enable debug tracing to see each iteration:

agent = Agent(
    chat_generator=chat_generator,
    tools=tools,
    system_prompt="You are a precise assistant.",
    max_iterations=5,
    raise_on_failure=False,
)
pipe = Pipeline()
pipe.add_component("agent", agent)
pipe.show()  # prints component graph

If the model ignores a tool, check the schema: missing required fields or vague descriptions cause silent falls back to text. Tighten the description and use explicit types.

This haystack 2.0 tool-calling agent tutorial gave you a runnable agent, a pipeline wrapper, and a verification assert. From here, swap the mock tools for real APIs, add a router component upstream, or stream tokens with generator.stream(). The agent loop is stable; your tools are the only moving part.