LlamaIndex PropertyGraphIndex with Claude on n4n.ai

Property graphs give you structured, queryable relationships that vector search alone cannot capture. LlamaIndex’s PropertyGraphIndex extracts entities and relations from documents, stores them in a graph database, and lets you traverse paths to answer multi-hop questions. This tutorial walks through building a working pipeline with Claude Sonnet 4 via n4n.ai, using Neo4j as the graph store and showing expected output at each stage.

Prerequisites

• Python 3.10+
• A Neo4j instance (local Docker or Aura)
• An n4n.ai API key (or any OpenAI-compatible endpoint)
• Documents to ingest — we’ll use a small corpus of technical markdown files

Install the dependencies:

pip install llama-index llama-index-graph-stores-neo4j llama-index-llms-anthropic neo4j python-dotenv

Create a .env file:

NEO4J_URI=bolt://localhost:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=your-password
N4N_API_KEY=your-n4n-key
N4N_BASE_URL=https://api.n4n.ai/v1

Project structure

property-graph-tutorial/
├── .env
├── data/
│   ├── api-design.md
│   ├── deployment.md
│   └── monitoring.md
├── build_graph.py
├── query_graph.py
└── utils.py

The data/ directory contains three markdown files about platform engineering — realistic, messy technical content that benefits from graph extraction.

Configure the LLM and embedding model

utils.py centralizes the model setup. We use Claude Sonnet 4 for extraction and a local embedding model for vector similarity fallback.

# utils.py
import os
from dotenv import load_dotenv
from llama_index.llms.anthropic import Anthropic
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import Settings

load_dotenv()

llm = Anthropic(
    model="claude-sonnet-4-20250514",
    api_key=os.getenv("N4N_API_KEY"),
    api_base=os.getenv("N4N_BASE_URL"),
    temperature=0.0,
    max_tokens=4096,
)

embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

Settings.llm = llm
Settings.embed_model = embed_model
Settings.chunk_size = 512
Settings.chunk_overlap = 50

The n4n.ai endpoint is OpenAI-compatible, so the Anthropic client works unchanged — just point api_base at the gateway. This also means automatic fallback if a provider degrades, and per-token usage metering without extra instrumentation.

Build the property graph index

build_graph.py loads documents, defines the extraction schema, and writes to Neo4j.

# build_graph.py
import os
from pathlib import Path
from dotenv import load_dotenv
from llama_index.core import SimpleDirectoryReader, PropertyGraphIndex
from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore
from llama_index.core.indices.property_graph import SchemaLLMPathExtractor
from utils import llm, embed_model

load_dotenv()

# 1. Load documents
documents = SimpleDirectoryReader("./data").load_data()
print(f"Loaded {len(documents)} documents")

# 2. Configure Neo4j graph store
graph_store = Neo4jPropertyGraphStore(
    username=os.getenv("NEO4J_USERNAME"),
    password=os.getenv("NEO4J_PASSWORD"),
    url=os.getenv("NEO4J_URI"),
    database="neo4j",
)

# 3. Define extraction schema — entities and relations we care about
kg_extractor = SchemaLLMPathExtractor(
    llm=llm,
    possible_entities=[
        "Service",
        "API",
        "Database",
        "Team",
        "DeploymentStrategy",
        "MonitoringTool",
        "Infrastructure",
        "Incident",
        "Runbook",
        "SLO",
    ],
    possible_relations=[
        "DEPENDS_ON",
        "OWNED_BY",
        "DEPLOYED_VIA",
        "MONITORED_BY",
        "HAS_SLO",
        "TRIGGERS",
        "RESOLVED_BY",
        "RUNS_ON",
        "VERSION_OF",
    ],
    strict=False,  # allow entities/relations outside the schema
    num_workers=2,
)

# 4. Build the index
index = PropertyGraphIndex.from_documents(
    documents,
    kg_extractors=[kg_extractor],
    property_graph_store=graph_store,
    show_progress=True,
)

print("Graph construction complete")
print(f"Nodes in graph: {index.property_graph_store.graph.nodes}")
print(f"Edges in graph: {index.property_graph_store.graph.edges}")

Run it:

python build_graph.py

Expected output (counts will vary with your corpus):

Loaded 3 documents
Graph construction complete
Nodes in graph: 47
Edges in graph: 62

Open Neo4j Browser at http://localhost:7474 and run:

MATCH (n)-[r]->(m) RETURN n, r, m LIMIT 50

You’ll see a graph like:

(:Service {name: "api-gateway"})-[:DEPENDS_ON]->(:Service {name: "auth-service"})
(:Service {name: "api-gateway"})-[:OWNED_BY]->(:Team {name: "platform"})
(:DeploymentStrategy {name: "blue-green"})-[:USED_BY]->(:Service {name: "api-gateway"})
(:MonitoringTool {name: "datadog"})-[:MONITORS]->(:Service {name: "api-gateway"})
(:Incident {name: "INC-2024-03-14"})-[:TRIGGERS]->(:Runbook {name: "latency-spike"})

The extractor pulls entities and relations directly from the markdown. strict=False lets the LLM introduce relevant types not in your seed list — useful for open-domain corpora.

Query the graph

query_graph.py demonstrates three query modes: graph traversal, vector similarity, and hybrid.

# query_graph.py
from llama_index.core import PropertyGraphIndex
from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore
from llama_index.core.retrievers import (
    VectorContextRetriever,
    KGTableRetriever,
)
from llama_index.core.query_engine import RetrieverQueryEngine
from utils import llm, embed_model
import os
from dotenv import load_dotenv

load_dotenv()

graph_store = Neo4jPropertyGraphStore(
    username=os.getenv("NEO4J_USERNAME"),
    password=os.getenv("NEO4J_PASSWORD"),
    url=os.getenv("NEO4J_URI"),
    database="neo4j",
)

# Load existing index from the graph store
index = PropertyGraphIndex.from_existing(
    property_graph_store=graph_store,
    llm=llm,
    embed_model=embed_model,
)

# 1. Graph traversal retriever — follows relationships
kg_retriever = KGTableRetriever(
    index=index,
    include_text=True,
    similarity_top_k=5,
    path_depth=2,  # how many hops to traverse
)

# 2. Vector retriever — falls back to semantic similarity
vector_retriever = VectorContextRetriever(
    index=index,
    similarity_top_k=5,
)

# 3. Hybrid query engine — combines both
from llama_index.core.retrievers import QueryFusionRetriever

hybrid_retriever = QueryFusionRetriever(
    [kg_retriever, vector_retriever],
    similarity_top_k=5,
    num_queries=1,  # set >1 for query rewriting
    mode="reciprocal_rerank",
)

query_engine = RetrieverQueryEngine.from_args(
    hybrid_retriever,
    llm=llm,
    response_mode="compact",
)

questions = [
    "What deployment strategy does the api-gateway use?",
    "Which team owns the service that depends on auth-service?",
    "What runbook handles latency spikes in the api-gateway?",
    "List all services monitored by datadog.",
]

for q in questions:
    print(f"\n{'='*60}")
    print(f"Q: {q}")
    print(f"{'='*60}")
    response = query_engine.query(q)
    print(f"A: {response}")
    print(f"\nSource nodes: {len(response.source_nodes)}")
    for i, node in enumerate(response.source_nodes[:3]):
        print(f"  [{i}] {node.node_type}: {node.text[:120]}...")

Run it:

python query_graph.py

Expected output:

============================================================
Q: What deployment strategy does the api-gateway use?
============================================================
A: The api-gateway uses a blue-green deployment strategy, as documented in the deployment guide. This strategy routes traffic between two identical environments to achieve zero-downtime releases.

Source nodes: 3
  [0] KG: The api-gateway is deployed via blue-green strategy...
  [1] KG: Deployment strategy blue-green is used by api-gateway...
  [2] Vector: The deployment guide recommends blue-green for stateless services...

============================================================
Q: Which team owns the service that depends on auth-service?
============================================================
A: The platform team owns the api-gateway, which depends on the auth-service for authentication and authorization.

Source nodes: 2
  [0] KG: api-gateway DEPENDS_ON auth-service
  [1] KG: api-gateway OWNED_BY platform team

============================================================
Q: What runbook handles latency spikes in the api-gateway?
============================================================
A: The latency-spike runbook is triggered by incidents involving the api-gateway. It includes steps to check Datadog APM traces, verify downstream service health, and scale the gateway horizontally.

Source nodes: 3
  [0] KG: INC-2024-03-14 TRIGGERS latency-spike runbook
  [1] KG: latency-spike runbook RESOLVED_BY platform team
  [2] Vector: Runbook for latency spikes covers gateway and downstream...

============================================================
Q: List all services monitored by datadog.
============================================================
A: Based on the knowledge graph, Datadog monitors: api-gateway, auth-service, payment-service, and notification-worker.

Source nodes: 4
  [0] KG: datadog MONITORS api-gateway
  [1] KG: datadog MONITORS auth-service
  [2] KG: datadog MONITORS payment-service
  [3] KG: datadog MONITORS notification-worker

The hybrid retriever shines on multi-hop questions. The second question — “which team owns the service that depends on auth-service?” — requires joining DEPENDS_ON and OWNED_BY edges. Pure vector search misses this; the graph traversal finds it in two hops.

Tune extraction quality

The default SchemaLLMPathExtractor works well, but three levers improve results:

1. Add few-shot examples to the extractor prompt:

from llama_index.core.indices.property_graph import SchemaLLMPathExtractor
from llama_index.core.prompts import PromptTemplate

custom_prompt = PromptTemplate("""
Extract entities and relations from the text. Use only the allowed types.

Allowed entities: {possible_entities}
Allowed relations: {possible_relations}

Examples:
Text: "The payments team owns the checkout-service which uses postgres."
Entities: [("checkout-service", "Service"), ("payments", "Team"), ("postgres", "Database")]
Relations: [("checkout-service", "OWNED_BY", "payments"), ("checkout-service", "USES", "postgres")]

Text: {text}
Entities:
Relations:
""")

kg_extractor = SchemaLLMPathExtractor(
    llm=llm,
    possible_entities=[...],
    possible_relations=[...],
    extract_prompt=custom_prompt,
    strict=False,
)

2. Increase path_depth on the retriever for deeper multi-hop questions (default is 2). Set to 3–4 for complex dependency chains.

3. Post-process with a validation pass — run a second LLM call to verify extracted triples against source text, filtering hallucinations.

Incremental updates

PropertyGraphIndex supports inserting new documents without rebuilding:

# add_documents.py
from llama_index.core import SimpleDirectoryReader
from build_graph import index  # reuse the built index

new_docs = SimpleDirectoryReader("./data/new").load_data()
for doc in new_docs:
    index.insert(doc)
print(f"Inserted {len(new_docs)} documents")

Deletion requires Cypher directly:

graph_store.query("MATCH (n:Service {name: 'legacy-api'}) DETACH DELETE n")

When to use PropertyGraphIndex vs. pure vector search

The graph shines when relationships are the answer. If your questions are “what does the doc say about X,” vector search is simpler and cheaper.

Cost and latency notes

• Extraction: one LLM call per chunk (512 tokens here). For 100 pages ~$2–4 with Claude Sonnet 4 via n4n.ai.
• Query: hybrid retriever makes 2–3 retrieval calls + 1 synthesis call. Typical latency 1.5–3s.
• Neo4j storage: negligible for <100k nodes. Use indexes on name and type properties for fast lookups.

CREATE INDEX entity_name IF NOT EXISTS FOR (n:__Entity__) ON (n.name);
CREATE INDEX entity_type IF NOT EXISTS FOR (n:__Entity__) ON (n.__entity_type__);

Next steps

• Add a text-to-Cypher layer for analytical queries (“count services per team”)
• Implement entity resolution to merge duplicates across documents
• Experiment with different extractors — LLMPathExtractor (no schema) or DynamicLLMPathExtractor (schema discovered per document)
• Hook up LlamaIndex agents with graph tools for autonomous multi-step research

The complete runnable code is in the structure above. Clone it, drop in your own markdown, and you have a queryable knowledge graph backed by Claude on n4n.ai in under an hour.