Legal research eats time because relevant precedent hides in long opinions. A case law research assistant llamaindex implementation lets you index thousands of court documents and ask natural-language questions with cited sources. This tutorial builds one from scratch using LlamaIndex and a local corpus of public-domain cases.
Prerequisites
- Python 3.10 or newer
pip install llama-index python-dotenv- An OpenAI API key (or any OpenAI-compatible endpoint)
- A directory
./casescontaining at least two plain-text case files (.txt). For testing, grab Marbury v. Madison and Brown v. Board of Education from CourtListener or Project Gutenberg and save them asmarbury.txtandbrown.txt.
Set your key in a .env file:
echo "OPENAI_API_KEY=sk-..." > .envLoading and parsing case text
LlamaIndex’s SimpleDirectoryReader handles directory traversal and file extraction. For .txt it returns one Document per file with file_path in metadata.
import os
from dotenv import load_dotenv
from llama_index.core import SimpleDirectoryReader
load_dotenv()
documents = SimpleDirectoryReader(
input_dir="cases",
required_exts=[".txt"],
recursive=False,
).load_data()
print(f"Loaded {len(documents)} documents")
for d in documents[:1]:
print(d.metadata, len(d.text))Expected output:
Loaded 2 documents
{'file_path': 'cases/marbury.txt'} 45231If you see zero documents, check the extension filter and that the path is relative to your run directory.
Chunking and LLM configuration
Case opinions are dense. Naive fixed-size splitting breaks sentences mid-holding. Use SentenceSplitter with a 1024-token window and 64-token overlap. Lock temperature to zero for deterministic legal summarization.
from llama_index.core import Settings
from llama_index.core.node_parser import SentenceSplitter
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)
Settings.node_parser = SentenceSplitter(chunk_size=1024, chunk_overlap=64)The default embedding model is OpenAI’s text-embedding-3-small. That is fine for a prototype; swap to a local BGE model later if egress cost becomes an issue.
Building the base vector index
VectorStoreIndex.from_documents parses the docs through Settings.node_parser, embeds each node, and builds an in-memory FAISS-style store.
from llama_index.core import VectorStoreIndex
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(
response_mode="compact",
similarity_top_k=4,
)response_mode="compact" concatenates the top nodes into one prompt, which preserves cross-reference context better than tree_summarize for short queries.
Querying with citations
A research tool is useless without traceability. The query response exposes source_nodes with scores and metadata.
response = query_engine.query(
"What did the Court hold about judicial review in Marbury v. Madison?"
)
print(response.response)
print("--- Sources ---")
for node in response.source_nodes:
print(node.metadata.get("file_path"), round(node.score, 3))Expected output shape:
The Court held that the Supreme Court has the power to review acts of Congress and declare them void if they conflict with the Constitution.
--- Sources ---
cases/marbury.txt 0.841
cases/marbury.txt 0.792
cases/brown.txt 0.411The score spread tells you the retrieval is discriminating. If brown.txt scores above 0.7 on a Marbury-specific question, tighten chunk size or add metadata filters.
Enriching with metadata
Plain file paths are weak signals. Attach case name, year, and jurisdiction before indexing so you can filter at query time.
for d in documents:
fname = d.metadata.get("file_path", "").lower()
if "marbury" in fname:
d.metadata.update({"case": "Marbury v. Madison", "year": 1803, "jurisdiction": "US-SCOTUS"})
elif "brown" in fname:
d.metadata.update({"case": "Brown v. Board", "year": 1954, "jurisdiction": "US-SCOTUS"})
index = VectorStoreIndex.from_documents(documents)Now constrain a query to a single jurisdiction:
from llama_index.core.vector_stores import MetadataFilters, FilterCondition
filters = MetadataFilters.from_dicts(
[{"key": "jurisdiction", "value": "US-SCOTUS"}],
condition=FilterCondition.AND,
)
filtered_engine = index.as_query_engine(
filters=filters, similarity_top_k=4
)This pushes the predicate to the vector store. With an in-memory store it post-filters; with pgvector or Chroma it can pre-filter for speed.
Turning it into a research agent
A case law research assistant llamaindex deployment becomes more useful when the model can decide to search, then synthesize. Wrap the query engine in a tool and give it to an OpenAI agent.
from llama_index.agent.openai import OpenAIAgent
from llama_index.core.tools import QueryEngineTool, ToolMetadata
tool = QueryEngineTool(
query_engine=filtered_engine,
metadata=ToolMetadata(
name="case_law_search",
description="Search indexed U.S. Supreme Court opinions for holdings and precedent",
),
)
agent = OpenAIAgent.from_tools([tool], llm=Settings.llm, verbose=True)
agent.chat("Find a case that limits congressional power under the commerce clause")Verbose mode prints the ReAct loop. You should see a Action: case_law_search step followed by an Observation containing retrieved text, then a final answer. The agent will not hallucinate cases outside the index because every claim must ground in tool output.
Model portability and fallback
The OpenAI class in LlamaIndex is just an OpenAI-compatible client. Point api_base at any compliant gateway to change providers without touching index code. For example, an OpenAI-compatible gateway such as n4n.ai exposes one endpoint for 240+ models and auto-falls back when a provider is degraded, so the LlamaIndex code above stays unchanged aside from the base URL and model string.
from llama_index.llms.openai import OpenAI
Settings.llm = OpenAI(
model="anthropic/claude-3-haiku",
api_base="https://api.n4n.ai/v1",
api_key=os.environ["N4N_API_KEY"],
temperature=0,
)This is the cleanest way to experiment with model routing while keeping the case law research assistant llamaindex stack stable.
Persistence and production notes
The in-memory index rebuilds on every restart. Persist it with a storage context:
from llama_index.core.storage.docstore import SimpleDocumentStore
from llama_index.core.storage.index_store import SimpleIndexStore
from llama_index.core.vector_stores import SimpleVectorStore
from llama_index.core import StorageContext
storage_context = StorageContext.from_defaults(
docstore=SimpleDocumentStore.from_persist_dir("storage"),
index_store=SimpleIndexStore.from_persist_dir("storage"),
vector_store=SimpleVectorStore.from_persist_dir("storage"),
)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
storage_context.persist("storage")For multi-user legal SaaS, move the vector store to pgvector and the docstore to Redis. Add row-level metadata (client_id) and enforce it in MetadataFilters per request—never trust the client to filter server-side alone.
Embedding cost scales with corpus size, not query volume. A 10k-opinion corpus at 1k tokens each is ~10M tokens; at $0.02/M that is $0.20 per full re-embed. Use incremental indexing via index.insert(document) to avoid re-embedding the whole corpus when a new ruling drops.
Finally, the case law research assistant llamaindex pattern here is the same one you would use for healthcare policy PDFs or regulatory filings—only the metadata schema changes. Build the filter layer early; retrofitting jurisdiction and date fields after a million nodes are embedded is painful.