An AI agent legal research citations pipeline needs more than a prompt and a hope. You need tool use, source-grounded retrieval, and a verification loop that rejects hallucinations before they reach a brief. This guide walks through building a minimal but production-shaped agent that searches real case law and returns verified citations.
Step 1: Configure the model gateway and client
Point your OpenAI-compatible client at a single endpoint that fronts multiple providers. For example, n4n.ai exposes one OpenAI-compatible endpoint covering 240+ models and automatically falls back when a provider is rate-limited or degraded, so you avoid writing your own retry logic. Set the base URL and key from environment variables.
import os
from openai import OpenAI
client = OpenAI(
base_url=os.environ["LLM_BASE_URL"], # e.g., https://api.n4n.ai/v1
api_key=os.environ["LLM_API_KEY"],
)
MODEL = os.environ.get("LLM_MODEL", "openai/gpt-4o-mini")Keep the model string configurable. Legal tasks benefit from stronger reasoning models; route to anthropic/claude-3-5-sonnet or similar by changing one env var. The gateway honors client routing directives, so the same code works across providers.
Why a gateway beats direct calls
Direct integration with one vendor breaks the moment that vendor has an outage or you hit a quota. A gateway that forwards provider cache-control hints also lets you reuse identical tool outputs across turns without re-paying for prompt tokens. n4n.ai forwards those hints, which matters when your agent repeats the same jurisdiction filter in a multi-step query.
Step 2: Connect to a real legal source API
Do not fake the corpus. CourtListener provides a free REST API over federal and state case law. A simple search endpoint returns opinions with citation metadata.
import requests
def search_case_law(query: str, limit: int = 5) -> list[dict]:
url = "https://www.courtlistener.com/api/rest/v4/search/"
params = {"q": query, "type": "o", "limit": limit}
token = os.environ.get("COURTLISTENER_TOKEN")
headers = {"Authorization": f"Token {token}"} if token else {}
resp = requests.get(url, params=params, headers=headers, timeout=10)
resp.raise_for_status()
return resp.json().get("results", [])The response includes citation, caseName, absolute_url, id, and court. Store these fields; they are your ground-truth citations. A typical result slice looks like:
{
"caseName": "Texas v. Johnson",
"citation": "491 U.S. 397",
"absolute_url": "https://www.courtlistener.com/opinion/111111/texas-v-johnson/",
"id": 111111,
"court": "supreme"
}Handle requests.HTTPError explicitly. CourtListener returns 429 when unauthenticated clients exceed rate limits; back off or set a token.
Step 3: Define agent tools with strict schemas
The agent must call your search function instead of guessing citations. Define a tool schema that matches the function signature exactly.
{
"type": "function",
"function": {
"name": "search_case_law",
"description": "Search CourtListener for case law matching a query. Returns citations and URLs.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Legal issue or keyword phrase"},
"limit": {"type": "integer", "default": 5}
},
"required": ["query"]
}
}
}Register the tool with the client call. The model will return a tool_calls block when it needs source data. Strict schemas prevent the model from inventing parameters like jurisdiction that your backend does not support.
Prompt the model to use the tool
The system prompt is the control surface. State the citation rule before the first user message.
SYSTEM_PROMPT = """You are a legal research agent. Every factual claim about case law
must be followed by a citation in the format (Case Name, Citation, URL).
You may only use citations returned by the search_case_law tool. If you have no
supporting citation, say 'Insufficient authority found.'"""Step 4: Implement the agent loop with citation enforcement
Run a loop that executes tools and feeds results back. Cap iterations to avoid runaway cost.
import json
SEARCH_TOOL = { /* JSON from Step 3 */ }
def run_agent(question: str, max_turns: int = 5):
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": question},
]
stored_results = []
for _ in range(max_turns):
resp = client.chat.completions.create(
model=MODEL, messages=messages, tools=[SEARCH_TOOL], tool_choice="auto"
)
msg = resp.choices[0].message
if not msg.tool_calls:
answer = msg.content
if verify_citations(answer, stored_results):
return answer
return "Citation verification failed; escalate to human."
messages.append(msg)
for call in msg.tool_calls:
if call.function.name == "search_case_law":
args = json.loads(call.function.arguments)
results = search_case_law(**args)
stored_results.extend(results)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(results),
})
return "Agent exceeded turn limit without resolved citations."This loop terminates when the model produces a final answer with no tool calls. The search_case_law output is raw JSON; the model extracts citations from it. If verification fails, the agent refuses to return unverified text.
Step 5: Validate returned citations against the source
The agent can still drift. Parse its final text for citation markers and confirm each appears in the tool results you stored. A simple post-check:
import re
def verify_citations(answer: str, stored: list[dict]) -> bool:
urls = set(re.findall(r'https?://\S+', answer))
valid = {r["absolute_url"] for r in stored}
# also accept courtlistener opinion ids as fallback
ids = {str(r["id"]) for r in stored}
mentioned_ids = set(re.findall(r'opinion/(\d+)/', answer))
missing = urls - valid
bad_ids = mentioned_ids - ids
return not missing and not bad_idsIf verify_citations fails, reject the answer and re-prompt with a stricter instruction or escalate to human review. This guard rail is non-negotiable in legal contexts. You can extend the checker to require the Bluebook-style citation field substring inside the answer.
Handling partial matches
Models sometimes truncate URLs. Normalize by checking that the opinion ID from the URL is present in stored. That catches most formatting slips without brittle string equality.
Step 6: Verify success
You need observable proof the AI agent legal research citations flow works end to end. Write a smoke test that runs a known query and asserts structure.
def test_agent_smoke():
q = "Does the First Amendment protect symbolic speech burning a flag?"
answer = run_agent(q)
assert "Texas v. Johnson" in answer or "Johnson" in answer
assert "http" in answer
print("Smoke test passed: agent returned citation-bearing answer")Run it locally:
export LLM_BASE_URL=https://api.n4n.ai/v1
export LLM_API_KEY=sk-...
export COURTLISTENER_TOKEN=optional
python -m pytest test_agent.py -qA successful run means the agent queried live case law, returned an answer containing a real URL from CourtListener, and the verification step found no orphan citations. For production, log the stored_results and the final answer to an append-only store for audit.
CI integration
Add the smoke test to your CI pipeline with a secret mask for the API key. Set a low max_turns in tests to keep runtime under a minute. If the gateway reports per-token usage metering, assert that the test call consumed fewer than a budgeted number of tokens.
Practical notes on reliability
Legal research tolerates no silent failure. Use the gateway’s per-token usage metering to track cost per query and set hard limits. When the primary model is throttled, the fallback at the gateway keeps the agent running without code changes.
Treat any AI agent legal research citations output as a draft. A licensed attorney must review before filing. The agent’s job is to surface relevant authority and format it correctly, not to render legal advice.
Build the agent as described, keep the verification loop strict, and you have a reproducible base for legal tech automation that will not invent cases.