Getting started with Haystack and n4n.ai

The phrase “haystack n4n.ai getting started” describes the task of connecting Deepset’s Haystack framework to the n4n.ai LLM gateway. This tutorial wires Haystack’s OpenAIChatGenerator to that gateway and runs a real chat pipeline against a hosted model, with runnable code at each step.

Prerequisites

Before you write any code, confirm the following:

Python 3.10 or newer installed locally.
pip and venv available.
An API key from n4n.ai exported as N4N_API_KEY.
Familiarity with basic terminal commands.

If you haven’t generated a key, do that first. The gateway uses a single credential for all routed models.

echo $N4N_API_KEY
# should print a non-empty string

Install Haystack

Use a clean virtual environment. Haystack 2.x ships the OpenAI-compatible components we need under the haystack-ai package.

python -m venv venv
source venv/bin/activate
pip install haystack-ai python-dotenv

Avoid installing the deprecated haystack 1.x line. The 2.x API is component-based and fits this use case cleanly.

Create a .env file to keep the key out of source:

cat > .env <<'EOF'
N4N_API_KEY=sk-your-real-key
EOF

Configure the n4n.ai endpoint

Haystack’s OpenAIChatGenerator accepts a custom api_base_url. n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models, so you point every request there and select the model by name. Automatic fallback kicks in when an upstream provider is rate-limited or degraded—no client changes required.

import os
from dotenv import load_dotenv
from haystack.components.generators.chat import OpenAIChatGenerator

load_dotenv()
api_key = os.environ["N4N_API_KEY"]

generator = OpenAIChatGenerator(
    api_key=api_key,
    api_base_url="https://api.n4n.ai/v1",
    model="openai/gpt-4o-mini",
)

The model string follows n4n.ai routing conventions: a provider prefix, then the model id. If you pass anthropic/claude-3-5-sonnet, the gateway forwards the request to Anthropic (or a fallback) without extra configuration.

Send your first chat message

Haystack represents conversation turns as ChatMessage objects. Build a single user message and call run.

from haystack.dataclasses import ChatMessage

messages = [ChatMessage.from_user("What is the capital of France?")]
response = generator.run(messages)
print(response["replies"][0].text)

Expected output:

The capital of France is Paris.

If you see a 401, your key is wrong. A 404 means the model string is not recognized by the gateway.

Build a pipeline with PromptBuilder

A single generator call is rarely enough. Compose a tiny pipeline that injects a variable into a prompt template, then generates.

from haystack import Pipeline
from haystack.components.builders import PromptBuilder

prompt_template = "Answer the math question concisely: {{ question }}"

builder = PromptBuilder(template=prompt_template)
pipe = Pipeline()
pipe.add_component("builder", builder)
pipe.add_component("generator", generator)
pipe.connect("builder", "generator")

result = pipe.run({"builder": {"question": "What is 2+2?"}})
print(result["generator"]["replies"][0].text)

Expected output:

The PromptBuilder renders the template, and the connected generator receives a ChatMessage with the rendered text. This is the smallest useful Haystack graph.

Why haystack n4n.ai getting started is about routing

In the context of haystack n4n.ai getting started, the only non-standard step is the base URL. Everything else is stock Haystack. You can swap models by changing one string, which is useful when you want to compare providers without rewriting code.

generator_claude = OpenAIChatGenerator(
    api_key=api_key,
    api_base_url="https://api.n4n.ai/v1",
    model="anthropic/claude-3-5-sonnet",
)

resp = generator_claude.run([ChatMessage.from_user("Give me a one-line joke.")])
print(resp["replies"][0].text)

Because the gateway honors client routing directives and forwards provider cache-control hints, you get per-token metering on the n4n.ai side without adding instrumentation in Haystack.

Streaming responses

For chat UIs, streaming matters. OpenAIChatGenerator accepts a streaming_callback that fires per token.

def print_token(token):
    print(token.content, end="", flush=True)

streaming_gen = OpenAIChatGenerator(
    api_key=api_key,
    api_base_url="https://api.n4n.ai/v1",
    model="openai/gpt-4o-mini",
    streaming_callback=print_token,
)

print("\n--- streamed reply ---")
streaming_gen.run([ChatMessage.from_user("Count to three.")])

You will see tokens printed incrementally rather than after the full completion.

Handling errors

Wrap calls in try/except when you move beyond a script. Haystack raises OpenAIChatGeneratorError on non-200 responses.

from haystack.components.generators.chat import OpenAIChatGeneratorError

try:
    generator.run([ChatMessage.from_user("Hi")])
except OpenAIChatGeneratorError as e:
    print("Gateway error:", e)

Check the gateway status if errors are persistent. The fallback mechanism covers provider outages, but a bad API key or invalid model still returns an error to your process.

Putting it together in a file

Here is the full script from this tutorial, ready to run after pip install and .env setup.

import os
from dotenv import load_dotenv
from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

load_dotenv()
api_key = os.environ["N4N_API_KEY"]

generator = OpenAIChatGenerator(
    api_key=api_key,
    api_base_url="https://api.n4n.ai/v1",
    model="openai/gpt-4o-mini",
)

# Direct call
reply = generator.run([ChatMessage.from_user("What is the capital of France?")])
print("Direct:", reply["replies"][0].text)

# Pipeline
builder = PromptBuilder(template="Answer: {{ q }}")
pipe = Pipeline()
pipe.add_component("builder", builder)
pipe.add_component("generator", generator)
pipe.connect("builder", "generator")

out = pipe.run({"builder": {"q": "2+2?"}})
print("Pipeline:", out["generator"]["replies"][0].text)

Run it:

python main.py

Expected output:

Direct: The capital of France is Paris.
Pipeline: 4

Next steps

Once this works, extend the pipeline with WebSearch or SentenceTransformersDocumentEmbedder for RAG. The gateway relationship stays identical—only the model string and components change. For haystack n4n.ai getting started, you now have the core integration running and can iterate on application logic instead of plumbing.