How to reduce hallucinations in agentic RAG systems

Agentic RAG systems hallucinate when the reasoning loop fills retrieval gaps with model priors instead of admitting missing context. To reduce hallucinations agentic RAG deployments need hard contracts between retrieval, tool use, and generation—not polite prompts asking the model to “stay grounded.” Below is an end-to-end build sequence you can ship this week.

Step 1: Constrain Retrieval to Return Provenance

Treat the vector store as a source of record, not a suggestion engine. Every retrieved chunk must carry an immutable identifier, the source document, and a relevance score. Drop anything below a threshold you set from real eval, not a guess.

from pydantic import BaseModel
from typing import List

class Chunk(BaseModel):
    doc_id: str
    chunk_id: str
    text: str
    score: float
    updated_at: int  # epoch ms, use for freshness filtering

def retrieve(query: str, k: int = 5, min_score: float = 0.78) -> List[Chunk]:
    # assume `index` is a hybrid vector + bm25 store with .search()
    hits = index.search(query, k=k)
    chunks = [Chunk(**h) for h in hits if h["score"] >= min_score]
    # optional: keep only the most recent 3 per doc_id to avoid stale contradictions
    latest = {}
    for c in chunks:
        if c.doc_id not in latest or c.updated_at > latest[c.doc_id].updated_at:
            latest[c.doc_id] = c
    return list(latest.values())

If retrieval returns zero chunks, the agent must branch to a “no context” path. Do not let it answer anyway. Stale or low-score hits are a primary driver of grounded-but-wrong outputs.

Step 2: Force Citations via Tool Contracts

The LLM should not emit final text without calling a cite tool that binds a claim to a chunk_id. Define the tool schema strictly and refuse to parse answers that bypass it.

{
  "name": "cite",
  "description": "Attach a sourced chunk to the pending answer.",
  "parameters": {
    "type": "object",
    "properties": {
      "chunk_id": {"type": "string"},
      "claim": {"type": "string"}
    },
    "required": ["chunk_id", "claim"]
  }
}

System prompt excerpt:

You are a research agent. For every factual claim in your answer you MUST call the cite tool with the supporting chunk_id before emitting the final message. If no chunk supports a claim, say "I don't have sourced data for that." Never reuse a chunk_id for a different claim.

Wire this with an OpenAI-compatible client:

from openai import OpenAI

client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="YOUR_KEY")
# n4n.ai exposes 240+ models behind one endpoint; pick a capable instruct model.
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=[{"type": "function", "function": cite_schema}],
    tool_choice="auto"
)

After the model responds, extract tool_calls and build a map from claim to chunk. If the final assistant message contains prose without corresponding citations, discard it.

Step 3: Validate Tool Outputs Before Synthesis

Never trust the model to map claims to real chunks. Write a validator that cross-checks every cite call against the retrieved set and verifies lexical or embedding overlap.

from sentence_transformers import SentenceTransformer
embed = SentenceTransformer("all-MiniLM-L6-v2")

def validate_citations(cites: List[dict], chunks: List[Chunk]) -> List[str]:
    valid = {c.chunk_id: c for c in chunks}
    errors = []
    for call in cites:
        cid = call["chunk_id"]
        if cid not in valid:
            errors.append(f"Citation {cid} not in retrieved context")
            continue
        # semantic overlap check: claim should be close to chunk text
        sim = embed.similarity(embed(call["claim"]), embed(valid[cid].text))
        if sim < 0.62:
            errors.append(f"Claim for {cid} weakly supported (sim={sim:.2f})")
    return errors

If errors is non-empty, return them to the agent as a tool result and force a retry. This single check eliminates fabricated references and paraphrased hallucinations that drift from source.

Step 4: Run a Separate Critic Pass

A single model generating and self-checking is biased toward confirming its own output. Spawn a critic using a cheaper, different model family. The critic only receives the citations and the draft answer, and outputs a strict verdict.

critic_prompt = """Given the cited chunks and the draft answer, respond with JSON:
{"verdict": "ok" | "unsupported", "reason": str}.
An answer is unsupported if any claim lacks direct backing in the cited chunk text."""

def critic_check(draft: str, chunks: List[Chunk], client) -> dict:
    for attempt in range(2):
        resp = client.chat.completions.create(
            model="claude-3-haiku",
            messages=[
                {"role": "system", "content": critic_prompt},
                {"role": "user", "content": f"CHUNKS: {chunks}\nANSWER: {draft}"}
            ],
            response_format={"type": "json_object"}
        )
        try:
            return json.loads(resp.choices[0].message.content)
        except json.JSONDecodeError:
            continue
    return {"verdict": "unsupported", "reason": "critic parse failure"}

Loop the critic up to two times. If it returns unsupported, strip the offending claim using the reason and re-run synthesis with the critic note attached. The critic model should be frozen during generator experiments to isolate variables.

Step 5: Route Models with Fallback to Avoid Degradation

Hallucinations spike when a provider returns truncated completions under load. Front your agent with a gateway that honors routing directives and fails over automatically. Using an OpenAI-compatible endpoint such as n4n.ai gives you automatic fallback when a provider is rate-limited or degraded, so the agent never falls back to guessing because a response got cut mid-tool-call.

# routing hint passed as header (illustrative; check your gateway's spec)
client.headers.update({"x-n4n-route": "primary:azure,fallback:bedrock"})

Set a max token guard and a parser that detects incomplete JSON or tool calls. On parse failure, retry with the next fallback model rather than patching the output. Add a timeout of 8s per generation; if the primary model lags, the fallback preserves the citation loop instead of timing out silently.

Verify Success

You cannot claim you reduce hallucinations agentic RAG quality without measurement. Build a golden set of 50 questions with known answers and human-annotated supporting chunks. Run the pipeline and compute:

• Citation precision: cited chunks that actually contain the claim.
• Unsupported claim rate: claims with no valid citation after critic.
• Silent failure rate: cases where agent answered with zero citations.

def eval_run(questions, pipeline):
    prec, unsupp, silent = [], [], 0
    for q in questions:
        out = pipeline(q)
        prec.append(out.citation_precision())
        unsupp.append(out.unsupported_rate())
        if not out.citations:
            silent += 1
    return {
        "mean_precision": sum(prec)/len(prec),
        "mean_unsupported": sum(unsupp)/len(unsupp),
        "silent_failures": silent
    }

A healthy system should show citation precision above 0.95 and unsupported rate below 0.02 on your golden set. If numbers regress after a model swap, the routing or critic step is misconfigured. Track per-token cost of the critic separately so the reliability gain is accountable.

Operational Notes

Log every retrieved chunk, tool call, and critic verdict. When a user flags a bad answer, replay the exact state to reproduce. Keep the critic model frozen while you iterate on the generator—changing both simultaneously hides which lever actually moved the needle.

Agentic RAG reliability is an engineering problem, not a prompt-tuning exercise. Enforce provenance, validate aggressively, and let a second model judge. Do that and you reduce hallucinations agentic RAG teams complain about to a manageable trickle.