LlamaIndex query engines vs LangChain retrievers

If you’re building retrieval-augmented generation, the llamaindex query engine vs langchain retriever decision shapes your entire data flow. LlamaIndex centers on structured query pipelines with built-in synthesis strategies; LangChain treats retrieval as a composable step inside a broader chain abstraction. Both work, but they optimize for different mental models and extension points.

Core abstraction differences

LlamaIndex’s QueryEngine is a self-contained pipeline: retrieve, optionally re-rank, then synthesize an answer. You instantiate an index, call as_query_engine(), and get an object that handles the full round trip. The engine owns the prompt templates, the response synthesis mode (tree summarize, compact, refine), and the citation logic.

from llama_index.core import VectorStoreIndex, QueryEngine

index = VectorStoreIndex.from_documents(docs)
query_engine = index.as_query_engine(
    similarity_top_k=5,
    response_mode="tree_summarize",
    node_postprocessors=[MetadataRelevancePostProcessor()]
)
response = query_engine.query("What were Q3 revenue drivers?")

LangChain’s Retriever is narrower by design — it only fetches documents. You compose it with a DocumentChain (stuff, map-reduce, refine) and an LLM to build the equivalent pipeline. The retriever implements get_relevant_documents() or the async ainvoke(); everything else lives upstream.

from langchain_core.retrievers import BaseRetriever
from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
qa_chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-4o-mini"),
    chain_type="stuff",
    retriever=retriever,
)
result = qa_chain.invoke({"query": "What were Q3 revenue drivers?"})

The practical difference: LlamaIndex gives you a configured answer machine out of the box. LangChain gives you a retriever you must wire into a chain. If you want to swap synthesis strategies in LlamaIndex, you change a parameter; in LangChain, you swap the chain type or write a custom CombineDocumentsChain.

Retrieval pipeline control

LlamaIndex exposes the retrieval stage through Retriever objects you can attach to a query engine or use standalone. The VectorIndexRetriever supports metadata filters, hybrid search (vector + keyword), and custom node post-processors — re-rankers, similarity cutoffs, deduplication — that run before synthesis.

from llama_index.core.retrievers import VectorIndexRetriever
from llama_index.core.postprocessor import (
    SimilarityPostprocessor,
    KeywordNodePostprocessor,
)

retriever = VectorIndexRetriever(
    index=index,
    similarity_top_k=20,
)
retriever = retriever.with_postprocessors([
    SimilarityPostprocessor(similarity_cutoff=0.75),
    KeywordNodePostProcessor(required_keywords=["revenue"]),
])
nodes = retriever.retrieve("Q3 revenue")

LangChain’s retriever interface is deliberately minimal. Advanced retrieval — hybrid search, re-ranking, query rewriting — lives in separate components you chain together: ContextualCompressionRetriever, MultiQueryRetriever, ParentDocumentRetriever. This composability is powerful but demands more boilerplate.

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import (
    LLMChainExtractor,
    EmbeddingsFilter,
)
from langchain_openai import OpenAIEmbeddings

base_retriever = vectorstore.as_retriever(search_kwargs={"k": 20})
compressor = EmbeddingsFilter(
    embeddings=OpenAIEmbeddings(),
    similarity_threshold=0.75,
)
compression_retriever = ContextualCompressionRetriever(
    base_retriever=base_retriever,
    base_compressor=compressor,
)
docs = compression_retriever.invoke("Q3 revenue")

LlamaIndex’s post-processor chain runs inside the query engine, so you get consistent behavior whether you call query() or aretrieve(). LangChain’s retriever composition is more explicit but can diverge if you use the retriever directly versus through a chain.

Query planning and synthesis

This is where LlamaIndex’s opinionated design pays off. The QueryEngine supports multiple response modes:

• Refine: Iteratively update an answer across retrieved nodes (good for long contexts)
• Compact: Pack nodes into fewer LLM calls (token-efficient)
• Tree summarize: Build a summary tree bottom-up (best for multi-document synthesis)
• Simple summarize: Single call with all context (fast, limited by context window)

You can also attach a SubQuestionQueryEngine that decomposes complex queries into parallel sub-queries, each with its own retriever, then synthesizes a final answer.

from llama_index.core.query_engine import SubQuestionQueryEngine
from llama_index.core.tools import QueryEngineTool

sub_engine = SubQuestionQueryEngine.from_defaults(
    query_engine_tools=[
        QueryEngineTool.from_defaults(
            query_engine=finance_engine,
            description="Financial reports",
        ),
        QueryEngineTool.from_defaults(
            query_engine=transcript_engine,
            description="Earnings calls",
        ),
    ],
)
response = sub_engine.query("Compare Q3 guidance vs actuals across segments")

LangChain handles multi-step reasoning through agents or explicit chain composition. The RetrievalQA chain types map to LlamaIndex’s modes: stuff ≈ compact, map_reduce ≈ tree summarize, refine ≈ refine. But there’s no built-in query decomposition — you build it with PlanAndExecute agents or custom logic.

from langchain.chains import MapReduceDocumentsChain, ReduceDocumentsChain
from langchain.chains.combine_documents.stuff import StuffDocumentsChain
from langchain.chains.llm import LLMChain
from langchain_core.prompts import PromptTemplate

map_prompt = PromptTemplate.from_template("Summarize: {docs}")
reduce_prompt = PromptTemplate.from_template("Synthesize: {docs}")

map_chain = LLMChain(llm=llm, prompt=map_prompt)
reduce_chain = LLMChain(llm=llm, prompt=reduce_prompt)

combine_documents_chain = StuffDocumentsChain(llm_chain=reduce_chain)
reduce_documents_chain = ReduceDocumentsChain(
    combine_documents_chain=combine_documents_chain,
)
map_reduce_chain = MapReduceDocumentsChain(
    llm_chain=map_chain,
    reduce_documents_chain=reduce_documents_chain,
)

LlamaIndex’s synthesis is more integrated; LangChain’s is more modular. If you need custom synthesis logic — streaming partial answers, emitting citations in a specific format, conditional logic based on retrieved metadata — LangChain’s chain composition gives you finer-grained control at the cost of more code.

Streaming and async ergonomics

Both frameworks support streaming, but the integration points differ. LlamaIndex’s QueryEngine exposes query() and aquery(); streaming requires a StreamingResponse object returned when you set streaming=True.

query_engine = index.as_query_engine(streaming=True, similarity_top_k=5)
streaming_response = query_engine.query("Explain the revenue variance")
for token in streaming_response.response_gen:
    print(token, end="", flush=True)

The streaming response includes source nodes and metadata after generation completes. You can also use async for with aquery() for async streaming.

LangChain’s streaming lives at the LLM level. The RetrievalQA chain doesn’t natively stream the final answer — you stream the underlying LLM and handle document retrieval separately. The typical pattern:

from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = ChatOpenAI(
    model="gpt-4o-mini",
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()],
)
qa_chain = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
# Streams tokens but retrieval happens synchronously first
result = qa_chain.invoke({"query": "Explain the revenue variance"})

For true async streaming with retrieval, you need astream() on a custom chain or use LangGraph. LlamaIndex’s integrated streaming is simpler for the common case; LangChain’s approach scales better when you need to interleave retrieval, tool calls, and generation in complex flows.

Ecosystem and integrations

LlamaIndex indexes are tightly coupled to its node/embedding model. Swapping vector stores (Pinecone, Weaviate, Qdrant, Postgres) is straightforward — the index abstraction handles it. But the query engine assumes LlamaIndex’s Node and Response objects. Using a non-LlamaIndex retriever requires implementing BaseRetriever and adapting output.

LangChain’s retriever interface is a protocol: any object with get_relevant_documents() or ainvoke() works. This makes it easier to drop in custom retrievers — Elasticsearch, SQL, API-backed search — without buying into a document model. The tradeoff: you lose the built-in post-processors and synthesis modes unless you reimplement them.

Both integrate with major observability tools (LangSmith, Arize, Langfuse). LlamaIndex’s callback system is newer; LangChain’s is more mature with broader community instrumentation.

Comparison table

Which to choose

Choose LlamaIndex QueryEngine when:

• You want a working RAG pipeline in 20 lines with sensible defaults
• Your queries benefit from tree-summarize or multi-document synthesis
• You need query decomposition across heterogeneous indexes (SQL + vector + graph)
• Streaming answers with citations out of the box matters
• Your team prefers configuration over composition

Choose LangChain Retriever + Chain when:

• You already have a LangChain/LangGraph stack and want consistent patterns
• Retrieval is one step in a larger agent workflow (tools, planning, memory)
• You need non-vector retrieval (SQL, API, graph) as a first-class retriever
• You want fine-grained control over every prompt and combination step
• You’re building custom synthesis logic that doesn’t map to existing modes

Hybrid approach: Use LlamaIndex for the retrieval+synthesis core, expose it as a LangChain-compatible retriever via LlamaIndexRetriever adapter, and compose it inside LangGraph for agentic workflows. This lets you keep LlamaIndex’s synthesis quality while using LangChain’s orchestration.

# Adapter pattern: LlamaIndex engine as LangChain retriever
from langchain_core.retrievers import BaseRetriever
from llama_index.core import QueryEngine

class LlamaIndexRetriever(BaseRetriever):
    query_engine: QueryEngine
    
    def _get_relevant_documents(self, query: str):
        response = self.query_engine.query(query)
        return [
            Document(page_content=n.text, metadata=n.metadata)
            for n in response.source_nodes
        ]

The llamaindex query engine vs langchain retriever choice isn’t permanent — both interoperate. Start with the abstraction that matches your current mental model, and migrate pieces when the other framework solves a specific pain point better.