Install and configure LlamaIndex for n4n.ai

This llamaindex install configure n4n.ai tutorial walks you through wiring LlamaIndex to an OpenAI-compatible gateway so you can swap models without rewriting application code. You will install the minimal dependencies, configure authentication, point the client at the gateway endpoint, and run a complete index-and-query cycle to verify the integration.

Prerequisites

• Python 3.10 or newer
• A n4n.ai API key (or any OpenAI-compatible endpoint and key)
• Access to pip and a virtual environment tool (venv, conda, or poetry)

Create and activate a clean environment before you begin:

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

Step 1: Install LlamaIndex and the OpenAI integration

LlamaIndex splits functionality into a core package and provider-specific integrations. For an OpenAI-compatible gateway you need the core, the OpenAI integration, and a few utilities for reading data.

pip install --upgrade pip
pip install "llama-index-core" "llama-index-llms-openai" "llama-index-embeddings-openai" "llama-index-readers-file"

Verify the imports work:

# verify_imports.py
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

print("Imports successful")

Run it:

python verify_imports.py

Step 2: Configure credentials and endpoint

Set the gateway base URL and your API key as environment variables. This keeps secrets out of source control and lets you switch environments without code changes.

export N4N_API_KEY="sk-your-key-here"
export N4N_BASE_URL="https://api.n4n.ai/v1"   # or your gateway's OpenAI-compatible endpoint

If you prefer a .env file, create one in the project root and load it with python-dotenv:

pip install python-dotenv

# config.py
import os
from dotenv import load_dotenv

load_dotenv()

API_KEY = os.getenv("N4N_API_KEY")
BASE_URL = os.getenv("N4N_BASE_URL", "https://api.n4n.ai/v1")

if not API_KEY:
    raise RuntimeError("N4N_API_KEY not set. Export it or add it to .env")

Step 3: Initialize the LLM and embedding models

LlamaIndex uses a global Settings object to hold the default LLM and embedding model. Point both at the gateway by passing api_base and api_key to the OpenAI-compatible classes.

# setup_models.py
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from config import API_KEY, BASE_URL

# Choose any model the gateway serves. The gateway handles routing and fallback.
llm = OpenAI(
    model="meta-llama/llama-3.1-70b-instruct",
    api_key=API_KEY,
    api_base=BASE_URL,
    temperature=0.1,
    max_tokens=1024,
)

embed_model = OpenAIEmbedding(
    model="text-embedding-3-small",
    api_key=API_KEY,
    api_base=BASE_URL,
)

Settings.llm = llm
Settings.embed_model = embed_model
Settings.chunk_size = 512
Settings.chunk_overlap = 50

print(f"LLM: {llm.model}")
print(f"Embedding: {embed_model.model_name}")
print(f"Endpoint: {BASE_URL}")

Run it to confirm the configuration:

python setup_models.py

You should see the model names and endpoint printed without errors.

Step 4: Ingest documents and build an index

Create a data/ directory and drop in a few text, PDF, or Markdown files. LlamaIndex’s SimpleDirectoryReader handles the parsing.

mkdir -p data
echo "LlamaIndex connects data to LLMs. It supports RAG, agents, and structured extraction." > data/intro.txt
echo "n4n.ai provides a single OpenAI-compatible endpoint for 240+ models with automatic fallback and per-token metering." > data/gateway.txt

Now build the vector index:

# build_index.py
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.core import Settings
from setup_models import llm, embed_model  # ensures Settings are applied

documents = SimpleDirectoryReader("data").load_data()
print(f"Loaded {len(documents)} documents")

index = VectorStoreIndex.from_documents(
    documents,
    llm=llm,
    embed_model=embed_model,
    show_progress=True,
)

# Persist to disk so you can reload without re-embedding
index.storage_context.persist(persist_dir="./storage")
print("Index built and persisted to ./storage")

Run it:

python build_index.py

Expect output showing the document count and a progress bar for embedding. The ./storage directory will contain the serialized index.

Step 5: Query the index

With the index persisted, you can load it and run queries without re-ingesting. This script demonstrates a basic retrieval-augmented generation (RAG) query.

# query_index.py
from llama_index.core import VectorStoreIndex, StorageContext, load_index_from_storage
from llama_index.core import Settings
from setup_models import llm, embed_model

storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(
    storage_context,
    llm=llm,
    embed_model=embed_model,
)

query_engine = index.as_query_engine(
    similarity_top_k=3,
    response_mode="compact",
)

questions = [
    "What does LlamaIndex do?",
    "How many models does the gateway serve?",
    "What is the fallback behavior?",
]

for q in questions:
    print(f"\nQ: {q}")
    response = query_engine.query(q)
    print(f"A: {response}")
    # Inspect source nodes if needed
    for node in response.source_nodes:
        print(f"  Source (score={node.score:.3f}): {node.text[:80]}...")

Run it:

python query_index.py

You should see answers grounded in the two documents you created, with source citations and similarity scores.

Step 6: Verify end-to-end success

A successful run produces three artifacts:

1. No import or authentication errors in any step.
2. A ./storage directory containing docstore.json, index_store.json, vector_store.json, and graph_store.json.
3. Coherent answers that reference your source documents, not hallucinated content.

Quick sanity check from the command line:

ls -la storage/
python -c "
from llama_index.core import StorageContext, load_index_from_storage
from setup_models import llm, embed_model
ctx = StorageContext.from_defaults(persist_dir='./storage')
idx = load_index_from_storage(ctx, llm=llm, embed_model=embed_model)
print('Index loaded, vector count:', idx.vector_store._data.embedding_dict.__len__())
"

If the vector count matches the number of chunks you embedded (roughly total_tokens / chunk_size), the pipeline is wired correctly.

Common issues and fixes

Next steps

• Swap models by changing the model parameter in OpenAI() and OpenAIEmbedding() — no other code changes required.
• Add metadata filtering by attaching metadata to documents before indexing and using MetadataFilters on the query engine.
• Enable streaming with query_engine.query(...).response_gen for lower perceived latency.
• Instrument usage — the gateway returns per-token usage in response headers; log them for cost tracking.
• Scale storage — replace the default in-memory vector store with PGVector, Pinecone, or Weaviate by passing a vector_store argument to VectorStoreIndex.from_documents.

You now have a minimal, production-ready LlamaIndex pipeline backed by an OpenAI-compatible gateway. The same pattern extends to agents, multi-modal indexes, and structured extraction workflows.