n4nAI
Open console →
Framework tutorialsTutorialLlamaIndex Query Engines for RAG

Build a citation-aware query engine in LlamaIndex

Hands-on llamaindex citation query engine tutorial: build a RAG pipeline with LlamaIndex that returns answers annotated with verifiable source citations.

n4n Team2 min read528 words

Audio narration

Coming soon — every post will get a voice note here.

This llamaindex citation query engine tutorial walks through building a retrieval-augmented generation (RAG) system that grounds responses in sourced documents. We’ll use LlamaIndex’s CitationQueryEngine to attach verifiable citations to generated answers, so your users can trace every claim back to the original text.

Prerequisites

  • Python 3.10+
  • llama-index (core) and llama-index-llms-openai installed
  • An OpenAI API key, or any OpenAI-compatible endpoint. If you want one endpoint that covers 240+ models with automatic fallback when a provider is rate-limited, n4n.ai exposes an OpenAI-compatible API you can drop into Settings.
  • A small corpus of text files in a data/ directory.
pip install llama-index llama-index-llms-openai llama-index-embeddings-openai

Load documents and build an index

Point SimpleDirectoryReader at your corpus. For deterministic results, fix the chunk size.

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex, Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")

docs = SimpleDirectoryReader("data/").load_data()
index = VectorStoreIndex.from_documents(docs, chunk_size=512)

If your files are absent, create a sample:

mkdir -p data
echo "LlamaIndex supports citation query engines for RAG." > data/a.txt
echo "Citations map generated text to source nodes." > data/b.txt

Configure the CitationQueryEngine

The citation engine subclasses the standard query engine. It post-processes the LLM output to inject markers like [1] and binds them to retrieved nodes.

from llama_index.core.query_engine import CitationQueryEngine

query_engine = CitationQueryEngine.from_args(
    index,
    similarity_top_k=3,
    citation_chunk_size=512,
    generate_localized_citations=True,
)

generate_localized_citations=True makes the engine pass cited chunks to the LLM prompt so the model stays grounded. Without it, you get less reliable markers.

Run a query and inspect output

Execute a query and print the response plus the source mapping.

response = query_engine.query("How does LlamaIndex handle citations in RAG?")
print(response.response)
print("---")
for i, src in enumerate(response.source_nodes):
    print(f"[{i+1}] score={src.score:.3f} file={src.node.metadata.get('file_name')}")
    print(src.node.get_content()[:120])

Expected output:

LlamaIndex handles citations in RAG by using a CitationQueryEngine that injects markers such as [1] into the response and binds them to retrieved source nodes [1][2].

---
[1] score=0.81 file=a.txt
LlamaIndex supports citation query engines for RAG.
[2] score=0.77 file=b.txt
Citations map generated text to source nodes.

The markers in the text correspond to the indexed source_nodes list, in order.

Extract structured citations

For a UI, you need a clean mapping. The response object exposes source_nodes with metadata. Build a payload:

def build_citations(response):
    out = []
    for node in response.source_nodes:
        out.append({
            "citation_id": node.id_,
            "file": node.metadata.get("file_name"),
            "score": node.score,
            "text": node.get_content()[:200],
        })
    return out

import json
print(json.dumps(build_citations(response), indent=2))

This yields JSON your frontend can render as footnotes.

How the citation parser binds markers

The engine prefixes each retrieved chunk with a numeric tag in the prompt, e.g., Context [1]: .... After generation, a regex extracts [n] spans and maps them to source_nodes[n-1]. If the model emits [0] or out-of-range, the parser drops it. Understanding this helps when citations look off.

import re
markers = re.findall(r"\[(\d+)\]", response.response)
valid = [int(m) for m in markers if 1 <= int(m) <= len(response.source_nodes)]

Streaming with citations

CitationQueryEngine does not natively stream markers token-by-token in current releases, but you can stream the underlying response and attach citations after. For production, generate the full response then stream the text while lazy-loading sources from a cache.

# Pattern: query, then stream stored string
def stream_response(text):
    for chunk in text.split(" "):
        yield chunk + " "

for tok in stream_response(response.response):
    print(tok, end="", flush=True)

Production considerations

  • Citation integrity: The LLM may hallucinate a [3] with no matching node. Validate markers against len(response.source_nodes) before rendering.
  • Chunk size: Smaller citation_chunk_size yields finer-grained citations but more nodes. Tune to your corpus.
  • Cost: Each citation query sends retrieved chunks to the LLM. Use a cheap model for drafting and a stronger one for final answer if needed.
  • Routing: If you use a gateway that honors client routing directives, you can pin citation generation to a specific model with low latency.

In this llamaindex citation query engine tutorial we kept the stack minimal, but the same pattern works with any BaseLLM implementation.

Custom prompt for stricter citations

Override the citation prompt to force the model to only cite provided context.

from llama_index.core.prompts import PromptTemplate

citation_prompt = PromptTemplate(
    "Given context: {context_str}\n"
    "Answer the question: {query_str}\n"
    "Only use citations [n] that map to context. If unsure, say 'no source'.\n"
)

query_engine = CitationQueryEngine.from_args(
    index,
    text_qa_template=citation_prompt,
    similarity_top_k=2,
)

This reduces unsupported claims in our testing on internal docs.

Evaluate with a quick test

Add a pytest to catch out-of-range markers in CI:

import re

def test_citations_in_range():
    resp = query_engine.query("What does LlamaIndex support?")
    ids = [int(m) for m in re.findall(r"\[(\d+)\]", resp.response)]
    assert all(1 <= i <= len(resp.source_nodes) for i in ids)

Expose as an HTTP endpoint

Wrap the engine in FastAPI to serve it:

from fastapi import FastAPI
app = FastAPI()

@app.post("/ask")
def ask(q: str):
    resp = query_engine.query(q)
    return {"answer": resp.response, "citations": build_citations(resp)}

Run with uvicorn main:app --reload. This closes the loop for a production RAG service.

You now have a runnable llamaindex citation query engine tutorial implementation: indexed docs, a citation-aware query engine, and a structured citation extractor. Swap the LLM backend to your preferred provider and ship it behind your API.

Tagsllamaindexcitationsquery-enginerag

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Framework tutorialsSub-question query engine in LlamaIndex explainedA practical guide to LlamaIndex's SubQuestionQueryEngine — when to use it, how it decomposes complex queries, and production patterns for multi-document RAG.Framework tutorialsHybrid search query engine in LlamaIndex tutorialBuild a production-ready hybrid search query engine in LlamaIndex combining vector similarity and BM25 keyword search with runnable code and tuning guidance.Framework tutorialsBuild a RAG query engine in LlamaIndex with n4n.aiStep-by-step llamaindex rag query engine tutorial: install LlamaIndex, load docs, configure LLM via n4n.ai, build vector index, and run queries.Framework tutorialsLlamaIndex VectorStoreIndex query engine explainedA practical llamaindex vectorstoreindex query engine tutorial: build the index, configure models, tune retrieval, run RAG queries, and avoid common pitfalls.Framework tutorialsLlamaIndex query pipelines vs query enginesCompare LlamaIndex query pipelines and query engines across capabilities, ergonomics, latency, and ecosystem to pick the right abstraction for your RAG system.Framework tutorialsLlamaIndex tree summarize vs refine response modeCompare LlamaIndex tree_summarize vs refine response modes for RAG — latency, cost, quality trade-offs, and when to use each.

All llamaindex query engines for rag posts →