Building a clinical notes Q&A system with LlamaIndex

Clinical notes qa llamaindex pipelines demand stricter grounding than generic chatbots because a hallucinated dosage kills trust instantly. This tutorial builds a runnable system that ingests synthetic clinic notes, indexes them with LlamaIndex, and answers questions with cited source spans.

Prerequisites

• Python 3.11 or newer
• llama-index-core, llama-index-llms-openai, llama-index-embeddings-huggingface
• A directory ./notes with at least two plain-text clinical notes (synthetic, no real PHI)
• An API key for an OpenAI-compatible inference service. If you want automatic fallback when a provider is degraded, point the client at n4n.ai’s OpenAI-compatible endpoint; it meters per token and forwards cache-control hints.

Step 1: Install dependencies

pip install llama-index-core llama-index-llms-openai llama-index-embeddings-huggingface

Step 2: Prepare synthetic notes and parse real formats

Create ./notes/note_001.txt:

Patient: SYN-001
Date: 2024-03-12
Chief Complaint: Hypertension follow-up
Assessment: BP 148/92. Started lisinopril 10mg daily.
Plan: Recheck in 4 weeks. Avoid potassium supplements.

And ./notes/note_002.txt:

Patient: SYN-002
Date: 2024-03-15
Chief Complaint: Type 2 diabetes
Assessment: HbA1c 8.1%. Metformin 500mg BID tolerated.
Plan: Increase to 1000mg BID. Eye exam referral.

Real clinical exports often arrive as CDA/HL7 XML. Flatten them before indexing:

import xml.etree.ElementTree as ET

def cda_to_text(path: str) -> str:
    tree = ET.parse(path)
    root = tree.getroot()
    # Collect all non-empty text nodes, ignoring namespaces loosely
    chunks = [t.text.strip() for t in root.iter() if t.text and t.text.strip()]
    return "\n".join(chunks)

# Usage: open("./notes/cda_003.xml").read() -> cda_to_text("./notes/cda_003.xml")

Step 3: Configure the models

We use a local embedding model to avoid sending note text to external embedding APIs, and an OpenAI-compatible LLM for synthesis.

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.llm = OpenAI(
    model="gpt-4o-mini",
    api_key="YOUR_KEY",
    base_url="https://api.n4n.ai/v1",  # optional gateway for fallback
    temperature=0.0,
)
Settings.chunk_size = 512
Settings.chunk_overlap = 64

temperature=0.0 reduces stochasticity. The chunk size bounds how much context the retriever returns per node.

Step 4: Load and index

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

docs = SimpleDirectoryReader("./notes").load_data()
index = VectorStoreIndex.from_documents(docs)

This builds an in-memory vector store. For production, persist it (see Step 8).

Step 5: Query with citations

LlamaIndex’s CitationQueryEngine forces the model to attach [1], [2] markers mapped to source nodes.

from llama_index.core.query_engine import CitationQueryEngine

query_engine = CitationQueryEngine.from_args(
    index,
    similarity_top_k=3,
    citation_chunk_size=512,
)

response = query_engine.query("What medication was started for SYN-001?")
print(str(response))

Expected output:

Lisinopril 10mg daily was started for SYN-001 [1].

Sources:
[1] note_001.txt: "Started lisinopril 10mg daily."

If you see an answer without a bracketed citation, the engine failed to ground it; treat that as a hard error.

Step 6: Enforce grounding programmatically

Wrap the query in a checker that rejects ungrounded responses:

import re

def grounded_answer(query_engine, question: str):
    resp = query_engine.query(question)
    text = str(resp)
    if not re.search(r"\[\d+\]", text):
        raise ValueError(f"Ungrounded answer: {text}")
    return text, resp.source_nodes

ans, nodes = grounded_answer(query_engine, "What was the diabetes plan for SYN-002?")
print(ans)

Expected:

Increase metformin to 1000mg BID and refer for eye exam [1].

Sources:
[1] note_002.txt: "Increase to 1000mg BID. Eye exam referral."

Step 7: Batch evaluation

Run a fixed set of questions and record citation coverage:

eval_qs = [
    "What was the BP reading for SYN-001?",
    "Which patient was prescribed metformin?",
    "Any potassium advice for SYN-001?",
]

for q in eval_qs:
    try:
        ans, _ = grounded_answer(query_engine, q)
        print(f"OK: {q}")
    except ValueError as e:
        print(f"FAIL: {e}")

This loop surfaces regressions before deploy.

Step 8: Persist the index

Avoid re-embedding on every restart:

index.storage_context.persist(persist_dir="./storage")

Reload later:

from llama_index.core import StorageContext, load_index_from_storage

ctx = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(ctx)

Production considerations

Clinical text is sensitive. Never log full note content with responses. Strip metadata from source nodes before returning to a frontend.

Use a gateway that supports per-token metering so you can attribute cost to each clinician session. If you built on n4n.ai, the usage endpoint returns token counts per request without extra instrumentation.

For larger corpora, replace SimpleDirectoryReader with a database connector and enable incremental indexing. LlamaIndex supports Document hashing to skip unchanged files.

The CitationQueryEngine adds latency because it performs a second LLM pass to insert markers. If latency bounds are tight, use response_mode="compact" with a prompt that demands citations, and validate with the regex above.

That’s a complete, runnable clinical notes qa llamaindex starter. Extend it with role-based access control and audit trails before any real patient data touches the pipeline.