When you’re facing LlamaIndex empty query results debugging, the instinct is to blame the retriever, but the root cause is usually a silent breakdown somewhere between document ingestion and the final LLM call. An empty response is a symptom, not a cause—most failures stem from missing nodes, misapplied filters, or an LLM that returned a null completion without raising.
1. Confirm the Index Actually Has Nodes
A VectorStoreIndex with zero documents will happily answer queries with an empty string. Before touching retrievers, inspect the docstore.
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader("data/").load_data()
index = VectorStoreIndex.from_documents(documents)
# Inspect what got indexed
nodes = index.docstore.get_all_nodes()
print(f"Indexed node count: {len(nodes)}")
for node_id, node in list(nodes.items())[:3]:
print(node_id, node.get_content()[:80])Common pitfall: SimpleDirectoryReader silently skips empty files or unsupported extensions. If len(nodes) is 0, check the path and file types. A relative path that resolves to a different directory in your deployment container is a classic source of confusion. Also verify that load_data() didn’t throw a partial read on a corrupted PDF—PyPDF can return empty text without error.
2. Reproduce Retrieval Without the LLM
The fastest way to isolate the problem is to bypass response synthesis entirely. Pull nodes directly from the retriever.
retriever = index.as_retriever(similarity_top_k=3)
nodes = retriever.retrieve("What is the refund policy?")
print(f"Retrieved {len(nodes)} nodes")
for n in nodes:
print(n.score, n.get_content()[:100])If this list is empty, your issue is purely in embedding or similarity search. During LlamaIndex empty query results debugging, this step separates retrieval bugs from LLM bugs. If you are using an async pipeline, call await retriever.aretrieve(...) in a coroutine—mixing sync and async entrypoints can return an empty coroutine object that serializes to nothing.
Check embedding model alignment. The embedding used at query time must match the one used at index time.
from llama_index.core import Settings
from llama_index.embeddings.openai import OpenAIEmbedding
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
# Rebuild index if you changed this after first buildIf you indexed with a local HuggingFace model and query with OpenAI, cosine distances will be meaningless and top_k will return nothing relevant—or nothing at all if a postprocessor filters by score. This mismatch is invisible until you print scores.
3. Check Chunk Size and Similarity Cutoffs
Default similarity_top_k is 2, which is fine, but a SimilarityPostprocessor with an aggressive cutoff will drop every node.
from llama_index.core.postprocessor import SimilarityPostprocessor
retriever = index.as_retriever(
similarity_top_k=5,
node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.8)]
)If all retrieved scores are below 0.8, you get an empty list. Lower the cutoff or remove the postprocessor during debugging. Tradeoff: lowering cutoff floods the LLM with weakly relevant context, increasing cost and hallucination risk. Another pitfall is chunk size: if you split documents into 2048-token chunks but your embedding model max input is 512, the embedder truncates silently and similarity degrades.
4. Inspect Metadata Filters
Metadata filters are a frequent culprit. A filter that matches no documents produces zero nodes and thus an empty answer.
from llama_index.core.vector_stores import MetadataFilters, FilterCondition
filters = MetadataFilters(
filters=[{"key": "doc_type", "value": "contract"}],
condition=FilterCondition.AND
)
retriever = index.as_retriever(filters=filters)Verify the metadata exists on your nodes. Print it:
for node in index.docstore.get_all_nodes().values():
print(node.metadata)If your ingestion pipeline didn’t set doc_type, the filter silently excludes everything. This is a design tradeoff: filters reduce noise but introduce a coupling between ingestion and query code that breaks silently. Use a default fallback filter or validate filter keys at startup.
5. Validate the Query Engine, Prompt, and Response Mode
Assuming nodes are retrieved, the response synthesizer may still emit nothing. The default ResponseSynthesizer uses a compact prompt; a custom prompt with a typo or wrong variable can yield empty output.
from llama_index.core.query_engine import RetrieverQueryEngine
query_engine = RetrieverQueryEngine.from_args(retriever)
response = query_engine.query("Explain clause 4")
print(response.response) # empty?A common thread in LlamaIndex empty query results debugging is ignoring the LLM layer and the response_mode. If you set response_mode="no_text", the engine deliberately returns no synthesized text. Similarly, streaming=True with a consumer that never iterates the generator will look empty.
query_engine = RetrieverQueryEngine.from_args(
retriever, response_mode="compact", streaming=False
)Test the LLM directly:
from llama_index.core import Settings
print(Settings.llm.complete("Say hello").text)If that returns empty, your Settings.llm is misconfigured—maybe max_tokens=0 or a broken API key that LlamaIndex swallows in older versions.
6. The LLM Call: Provider Errors and Silent Failures
LlamaIndex does not always raise on provider errors. A 429 from OpenAI can be caught and result in an empty completion depending on your version and error handling. If you’re routing through a single provider and hitting rate limits, the SDK may surface an empty response or raise. A gateway such as n4n.ai that offers automatic fallback across 240+ models on one OpenAI-compatible endpoint can prevent transient provider degradation from masquerading as a retrieval bug.
To confirm, wrap the query in verbose logging:
import logging
logging.basicConfig(level=logging.DEBUG)Look for llm_response events with empty response text. If the raw HTTP call failed, you’ll see stack traces; if it succeeded with empty choices, the model returned nothing. Some models return finish_reason="length" with truncated empty content when max_tokens is too low.
7. Add Callback Tracing
LlamaIndex has a callback system that shows exactly where nodes disappear.
from llama_index.core import set_global_handler
set_global_handler("simple") # prints events to stdoutYou’ll see retrieve and synthesize events with payload sizes. If retrieve shows 0 nodes, the problem is upstream. If synthesize shows nodes but no output, the LLM is the issue. For production, use the LlamaDebugHandler to capture event metadata programmatically instead of stdout spam.
8. Systematic Debugging Checklist
Ordered path for LlamaIndex empty query results debugging:
- Print
len(index.docstore.get_all_nodes())— must be > 0. - Call
retriever.retrieve(query)and print lengths/scores. - Remove all postprocessors and filters temporarily.
- Print
node.metadatato confirm filter keys exist. - Call
Settings.llm.complete("test")to verify the model responds. - Enable
set_global_handler("simple")and watch event payloads. - Check embedding model name in
Settings.embed_modelmatches index build. - Lower
similarity_top_kto 1 and cutoff to 0.0 to force at least one node. - Verify
response_modeis not"no_text"andstreamingis handled.
If step 8 still yields nothing, your vector store backend (e.g., Chroma, Pinecone) may have a connection issue returning empty results without error.
9. Tradeoffs: Over-Retrieval vs Empty Results
The naive fix is to crank similarity_top_k to 20 and drop cutoffs. That guarantees non-empty responses but increases token spend and introduces distractors. A better approach is to log score distributions during development:
scores = [n.score for n in retriever.retrieve("query")]
print(sorted(scores)[:10])Set your cutoff at the knee of that distribution. This keeps signal high without silent failures. Remember that embedding similarity is not calibrated probability; absolute thresholds are environment-specific.
10. When to Suspect the Vector Store
If you use a remote vector DB, an empty result can come from an expired credential or a mismatched namespace. LlamaIndex passes the query through; the DB returns []. Always run a direct client query:
# Example with chroma
import chromadb
client = chromadb.PersistentClient(path="chroma/")
col = client.get_collection("default")
print(col.query(query_texts=["test"], n_results=1))If that returns empty, the bug is in the DB, not LlamaIndex. For Pinecone, check the active index host and namespace string—an empty namespace query returns no error and no matches.
11. Node Content Truncation and Hidden Empty Strings
Sometimes a node is retrieved but its get_content() returns a string that is whitespace only because the original PDF extracted as \n\n. The LLM receives “empty” context and may answer with nothing or “I don’t know”. Add a guard during ingestion:
from llama_index.core.node_parser import SentenceSplitter
parser = SentenceSplitter(chunk_size=512, chunk_overlap=50)
nodes = parser.get_nodes_from_documents(documents)
nodes = [n for n in nodes if n.get_content().strip()]
print(f"After stripping: {len(nodes)} nodes")This prevents silent empty nodes from entering the index. It trades a small amount of ingestion time for far easier debugging later.
Empty responses are never random. They are the terminal state of a pipeline that dropped context somewhere. Walk the ordered path above and you’ll find the break in under ten minutes.