This hands-on tutorial builds a haystack agent pipeline n4n.ai function calling example from scratch. You will define Python tools, wire Haystack 2.0’s Agent into a Pipeline, and watch multi-step reasoning execute against a live LLM through an OpenAI-compatible gateway.
Prerequisites
- Python 3.10 or newer
haystack-ai(2.0+),openai,python-dotenv- An API key for a model gateway. We use n4n.ai’s OpenAI-compatible endpoint, which fronts 240+ models and handles provider fallback automatically.
pip install haystack-ai openai python-dotenvCreate a .env file in your project root:
N4N_API_KEY=your_key_hereVerify your install:
import haystack
print(haystack.__version__) # expect 2.xDefine the tools
Haystack treats any function decorated with @tool as a callable the model can request. The docstring becomes the tool description, so write it precisely. The type hints define the schema.
from haystack.tools import tool
@tool
def get_weather(city: str) -> str:
"""Return current weather summary for a given city."""
# Stand-in for a real API call
return f"Sunny, 22°C in {city}"
@tool
def multiply(a: int, b: int) -> int:
"""Multiply two integers and return the product."""
return a * bTest the tools locally before involving the model:
print(get_weather("Berlin")) # Sunny, 22°C in Berlin
print(multiply(6, 7)) # 42Configure the chat generator
Haystack’s OpenAIChatGenerator speaks the OpenAI chat protocol. Point api_base at the gateway and pass your key. The model string can reference any model the gateway routes to.
import os
from dotenv import load_dotenv
from haystack.components.generators.chat import OpenAIChatGenerator
load_dotenv()
generator = OpenAIChatGenerator(
model="openai/gpt-4o-mini",
api_key=os.environ["N4N_API_KEY"],
api_base="https://api.n4n.ai/v1",
generation_kwargs={"temperature": 0}
)Setting temperature to 0 keeps tool-call parsing deterministic, which matters when you are debugging agent traces.
Build the agent
The Agent component wraps the generator and the tool list. It runs the loop: send messages, get a tool call, execute locally, feed the result back, repeat until a final answer.
from haystack.components.agents import Agent
agent = Agent(
chat_generator=generator,
tools=[get_weather, multiply],
prompt="You are a concise assistant. Use the provided tools when needed."
)The prompt here is the system instruction. Keep it explicit about calling tools; vague system prompts produce lazy agents.
Wire it into a pipeline
A pipeline lets you compose the agent with other components. For this tutorial we add a single agent node and run it.
from haystack import Pipeline
from haystack.dataclasses import ChatMessage
pipeline = Pipeline()
pipeline.add_component("agent", agent)
query = "What is the weather in Lisbon and what is 6 times 7?"
result = pipeline.run(
data={"agent": {"messages": [ChatMessage.from_user(query)]}}
)The data dict mirrors the agent’s run signature: the key is the component name, the value is its input parameters.
Run and inspect output
Execute the script. The agent should emit two tool calls and then a synthesized answer.
messages = result["agent"]["messages"]
for m in messages:
print(f"{m.role}: {m.content}")
print("FINAL:", messages[-1].content)Expected console output (wording may vary):
user: What is the weather in Lisbon and what is 6 times 7?
assistant: <tool_call get_weather city='Lisbon'>
tool: Sunny, 22°C in Lisbon
assistant: <tool_call multiply a=6 b=7>
tool: 42
assistant: The weather in Lisbon is sunny, 22°C. 6 times 7 is 42.
FINAL: The weather in Lisbon is sunny, 22°C. 6 times 7 is 42.Because n4n.ai forwards provider cache-control hints, you can annotate static prompt prefixes with cache_control to cut token cost on repeated agent steps without changing Haystack code.
Add a second pipeline stage
Real systems rarely stop at the agent. Suppose you want to extract structured fields from the agent’s answer. Add a PromptBuilder and a second generator:
from haystack.components.builders import PromptBuilder
template = "Summarize the following into JSON with keys 'weather' and 'math':\n{{answer}}"
builder = PromptBuilder(template=template)
pipeline.add_component("builder", builder)
pipeline.add_component("summarizer", generator)
pipeline.connect("agent.messages", "builder.answer")
pipeline.connect("builder.prompt", "summarizer.prompt")
result = pipeline.run(
data={"agent": {"messages": [ChatMessage.from_user("Weather in Paris and 9*8?")]}}
)
print(result["summarizer"]["replies"][0].content)This shows the haystack agent pipeline function calling pattern scaling beyond a single node: the agent resolves tools, then a downstream component reformats.
Debugging tool calls
When the agent misbehaves, print the raw tool calls. Haystack attaches them to assistant messages as tool_calls.
for m in result["agent"]["messages"]:
if m.role == "assistant" and m.tool_calls:
for tc in m.tool_calls:
print(tc.name, tc.arguments)If arguments are malformed, tighten your type hints or docstring. The model infers schema from those alone.
Production considerations
Tool execution can raise. Wrap tool bodies in try/except and return a string error; the agent will reason about it.
@tool
def get_weather(city: str) -> str:
"""Return current weather summary for a given city."""
try:
# real network call here
return f"Sunny, 22°C in {city}"
except Exception as e:
return f"error: {e}"On the model side, if a provider is rate-limited, the gateway’s automatic fallback switches to a healthy endpoint without code changes. You keep the same api_base and model name.
Recap
- Define tools with
@tooland precise docstrings. Agentin Haystack 2.0 handles the function-calling loop natively.- Pipelines compose agents with builders and generators.
- Using an OpenAI-compatible gateway keeps your code provider-agnostic.
The haystack agent pipeline function calling approach separates reasoning (LLM) from execution (local Python), which is the right boundary for most backend integrations.