Build your first LlamaIndex app with n4n.ai

This llamaindex first app n4n.ai tutorial walks you through standing up a minimal retrieval-augmented generation (RAG) pipeline against n4n.ai’s OpenAI-compatible inference endpoint. You’ll go from an empty virtualenv to a working query engine that answers questions from a local text file, with real per-token usage metering on every call.

Prerequisites

• Python 3.10 or newer
• pip and a fresh virtual environment
• An API key from n4n.ai (set as N4N_API_KEY in your environment)
• Basic familiarity with Python and the shell

If you don’t have an n4n.ai key yet, sign up and create one; the gateway exposes a single /v1 OpenAI-compatible endpoint that fronts 240+ models, so the same code works if you later switch model families.

Step 1: Install dependencies

Create and activate a venv, then install LlamaIndex and a small env loader:

python -m venv .venv
source .venv/bin/activate
pip install llama-index python-dotenv

LlamaIndex splits into core and provider packages; the meta llama-index package pulls in the OpenAI LLM and embedding integrations we need.

Step 2: Configure the environment file

Store credentials outside source control:

echo "N4N_API_KEY=sk-..." > .env
echo "DATA_FILE=./docs/seed.txt" >> .env

We’ll load this with python-dotenv so the key never touches your codebase.

Step 3: Point LlamaIndex at n4n.ai

LlamaIndex’s OpenAI client is just an OpenAI-compatible HTTP client. Swap the base_url and key, and you’re talking to n4n.ai instead of OpenAI proper.

# config.py
import os
from dotenv import load_dotenv
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

load_dotenv()

Settings.llm = OpenAI(
    api_key=os.getenv("N4N_API_KEY"),
    base_url="https://api.n4n.ai/v1",
    model="openai/gpt-4o-mini",
)

Settings.embed_model = OpenAIEmbedding(
    api_key=os.getenv("N4N_API_KEY"),
    base_url="https://api.n4n.ai/v1",
    model="openai/text-embedding-3-small",
)

The openai/ prefix is a routing hint n4n.ai honors; because the gateway fronts 240+ models with automatic fallback when a provider is rate-limited or degraded, you can change that string to anthropic/claude-3-haiku later without touching client logic.

Step 4: Create a sample document

We need something to index. Drop a short text file in docs/:

mkdir -p docs
cat > docs/seed.txt <<'EOF'
Our deployment runs as a containerized service behind a load balancer.
We scale horizontally based on p95 latency, not CPU.
Secrets are injected at runtime from a vault, never baked into images.
Rollbacks are triggered automatically if error rate exceeds 2% post-deploy.
EOF

Step 5: Build the index

Now load the document and construct a vector index. LlamaIndex handles chunking and embedding transparently.

# build_index.py
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from config import Settings  # noqa: F401  (applies Settings side effects)

docs = SimpleDirectoryReader(
    input_dir="docs", required_exts=[".txt"]
).load_data()

index = VectorStoreIndex.from_documents(docs)
index.storage_context.persist(persist_dir="storage")
print(f"Indexed {len(docs)} document(s).")

Run it:

python build_index.py

Expected output:

Indexed 1 document(s).

A storage/ directory now holds the vector store and metadata. Re-running this is idempotent; for production you’d persist to a real vector DB, but the local default is fine for a first app.

Step 6: Query the index

Spin up a query engine and ask a question grounded in the text:

# query.py
from llama_index.core import StorageContext, load_index_from_storage
from config import Settings  # noqa: F401

storage_context = StorageContext.from_defaults(persist_dir="storage")
index = load_index_from_storage(storage_context)

query_engine = index.as_query_engine()
response = query_engine.query(
    "How are rollbacks triggered in our deployment?"
)

print("ANSWER:", response.response)
print("SOURCES:", [n.get_content()[:60] for n in response.source_nodes])

Run:

python query.py

Expected output (wording may vary slightly by model):

ANSWER: Rollbacks are triggered automatically if the error rate exceeds 2% after deployment.
SOURCES: ['Our deployment runs as a containerized service behind a load b...']

The answer is retrieved from the indexed chunk, not from the model’s training data. That’s the core RAG loop.

Step 7: Inspect token usage

n4n.ai meters per-token usage on every request. LlamaIndex surfaces the underlying OpenAI-style usage payload in the response metadata:

# extend query.py
print("USAGE:", response.metadata.get("usage"))

Typical output:

USAGE: {'prompt_tokens': 142, 'completion_tokens': 18, 'total_tokens': 160}

Those counts reflect the retrieval context plus the generated answer. Because n4n.ai forwards provider cache-control hints, repeated queries against the same large context can show reduced prompt token billing when the upstream provider supports prompt caching—no client change required.

Step 8: Swap models without code changes

The whole point of routing through a gateway is optionality. Edit config.py and change the LLM model string:

Settings.llm = OpenAI(
    api_key=os.getenv("N4N_API_KEY"),
    base_url="https://api.n4n.ai/v1",
    model="anthropic/claude-3-haiku",
)

Re-run query.py. The same VectorStoreIndex and query engine work unchanged. If the primary provider behind that route is degraded, n4n.ai’s automatic fallback shifts traffic to a healthy equivalent so your app keeps serving.

Common pitfalls

Embeddings mismatch. If you only override Settings.llm and forget Settings.embed_model, LlamaIndex calls OpenAI directly for embeddings, leaking your key and breaking the unified billing view. Always set both.

Model name prefixes. n4n.ai uses slash-separated ownership prefixes (openai/, anthropic/, meta/). Passing a bare gpt-4o-mini may work via default routing, but being explicit avoids ambiguity.

Chunk size. Default chunk size is 1024 tokens. For short docs it doesn’t matter; for long ones, tune Settings.chunk_size and Settings.chunk_overlap before indexing.

Where to go next

You now have a runnable llamaindex first app n4n.ai tutorial baseline: document loading, embedding, vector indexing, and grounded querying through a single OpenAI-compatible endpoint. From here, replace SimpleDirectoryReader with a database connector, add a streaming response via index.as_query_engine(streaming=True), or persist to a managed vector store like pgvector. The client code stays identical as long as n4n.ai remains your base_url.