LangChain vs LlamaIndex: chunking and indexing strategies

Both frameworks solve retrieval-augmented generation, but they diverge sharply on how they think about documents. LangChain treats chunking as a preprocessing step you configure once. LlamaIndex treats it as a first-class retrieval primitive with pluggable strategies, node parsers, and a dedicated index abstraction. That philosophical difference cascades into API design, retrieval quality, and how much control you actually have when things go wrong.

Chunking philosophy and APIs

LangChain exposes chunking through TextSplitter subclasses — RecursiveCharacterTextSplitter, TokenTextSplitter, MarkdownHeaderTextSplitter, and a handful of others. You instantiate a splitter, call split_documents(), and get a flat list of Document objects with page_content and metadata. The metadata carries chunk_index and start_index if you enable it, but there is no built-in concept of parent-child relationships or hierarchical retrieval.

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
    separators=["\n\n", "\n", " ", ""],
    length_function=len,
)
chunks = splitter.split_documents(documents)

LlamaIndex centers chunking on NodeParser classes that produce Node objects — each node knows its source_node (the parent), its relationships (prev/next/parent/child), and can carry arbitrary metadata. The SentenceSplitter is the workhorse, but SemanticSplitterNodeParser uses embedding similarity to find natural boundaries, and HierarchicalNodeParser builds a tree of chunks at multiple granularities in one pass.

from llama_index.core.node_parser import SentenceSplitter, SemanticSplitterNodeParser
from llama_index.embeddings.openai import OpenAIEmbedding

# Simple sentence-aware splitting
parser = SentenceSplitter(chunk_size=1024, chunk_overlap=200)
nodes = parser.get_nodes_from_documents(documents)

# Semantic splitting — boundaries where embedding distance spikes
embed_model = OpenAIEmbedding()
semantic_parser = SemanticSplitterNodeParser(
    buffer_size=1,
    breakpoint_percentile_threshold=95,
    embed_model=embed_model,
)
semantic_nodes = semantic_parser.get_nodes_from_documents(documents)

The semantic splitter is genuinely useful for technical docs where fixed-size chunks split mid-concept. It costs an embedding call per candidate boundary, so factor that into ingestion latency.

Indexing abstractions

LangChain’s VectorStore interface is deliberately thin: add_documents, similarity_search, max_marginal_relevance_search. The vector store handles indexing; LangChain does not. You choose FAISS, Chroma, Pinecone, Weaviate, Qdrant, or pgvector, and LangChain passes your chunks through. There is no index-level metadata, no hybrid search coordination, and no built-in reranking — you compose those yourself.

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
vectorstore = FAISS.from_documents(chunks, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 6})

LlamaIndex introduces VectorStoreIndex, SummaryIndex, TreeIndex, KeywordTableIndex, and KnowledgeGraphIndex as distinct index types, each with different retrieval semantics. VectorStoreIndex wraps a vector store but adds a VectorIndexRetriever that supports similarity_top_k, vector_store_query_mode (default, hybrid, sparse), and alpha for hybrid weighting. You can also attach a MetadataReplacementPostProcessor to swap node content with the original document text at query time — critical for citation accuracy.

from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.faiss import FaissVectorStore
import faiss

faiss_index = faiss.IndexFlatL2(1536)
vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

index = VectorStoreIndex(nodes, storage_context=storage_context)
retriever = index.as_retriever(
    similarity_top_k=6,
    vector_store_query_mode="hybrid",
    alpha=0.5,
)

LlamaIndex also supports PropertyGraphIndex for entity-relationship extraction, which LangChain has no equivalent for without building a custom graph pipeline.

Retrieval composition and reranking

LangChain expects you to chain retrievers manually. EnsembleRetriever combines BM25 and vector search with reciprocal rank fusion, but you wire the BM25 retriever yourself. Reranking means wrapping a CrossEncoder or calling Cohere/Jina rerank endpoints in a custom Runnable.

from langchain.retrievers import EnsembleRetriever
from langchain_community.retrievers import BM25Retriever

bm25_retriever = BM25Retriever.from_documents(chunks)
bm25_retriever.k = 6

ensemble = EnsembleRetriever(
    retrievers=[vectorstore.as_retriever(search_kwargs={"k": 6}), bm25_retriever],
    weights=[0.6, 0.4],
)

LlamaIndex bakes hybrid retrieval and reranking into the query engine. QueryEngine accepts node_postprocessors — SimilarityPostprocessor, KeywordNodePostprocessor, SentenceTransformerRerank, CohereRerank, LLMRerank — that run after initial retrieval but before synthesis. The RetrieverQueryEngine also supports response_mode (compact, refine, tree_summarize, simple_summarize) which controls how retrieved nodes are fed to the LLM.

from llama_index.core.postprocessor import SentenceTransformerRerank, SimilarityPostprocessor
from llama_index.core.query_engine import RetrieverQueryEngine

reranker = SentenceTransformerRerank(model="cross-encoder/ms-marco-MiniLM-L-6-v2", top_n=3)
similarity_filter = SimilarityPostprocessor(similarity_cutoff=0.7)

query_engine = RetrieverQueryEngine.from_args(
    retriever,
    node_postprocessors=[similarity_filter, reranker],
    response_mode="compact",
)
response = query_engine.query("How does the auth flow handle token refresh?")

This composition model is more opinionated but eliminates boilerplate when you need hybrid search + rerank + citation-aware synthesis.

Metadata handling and filtering

LangChain stores metadata as a flat dict on each Document. Filtering at query time depends entirely on the underlying vector store’s filter parameter — syntax varies by provider. There is no framework-level metadata schema or validation.

LlamaIndex nodes carry metadata plus excluded_embed_metadata_keys and excluded_llm_metadata_keys so you control what goes into embeddings versus what the LLM sees. MetadataFilters and ExactMatchFilter/MetadataFilter provide a unified filter DSL that translates to the vector store’s native syntax.

from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

filters = MetadataFilters(
    filters=[ExactMatchFilter(key="source", value="api-docs/v2")]
)
retriever = index.as_retriever(similarity_top_k=6, filters=filters)

This matters when you have multi-tenant data or versioned documentation and need consistent filtering across vector stores.

Ingestion pipelines and idempotency

LangChain has no built-in ingestion pipeline. You write a script that loads, splits, embeds, and upserts. Deduplication is manual — typically hashing content and checking the vector store before insert.

LlamaIndex provides IngestionPipeline with Cache (in-memory, Redis, or disk) and Docstore for tracking processed documents. You define transformations (splitters, extractors, embedding) once and run pipeline.run(documents=docs). The pipeline skips unchanged documents by content hash and updates only modified nodes.

from llama_index.core.ingestion import IngestionPipeline
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core.extractors import TitleExtractor
from llama_index.core.cache import IngestionCache

pipeline = IngestionPipeline(
    transformations=[
        SentenceSplitter(chunk_size=1024, chunk_overlap=200),
        TitleExtractor(),
        embed_model,
    ],
    cache=IngestionCache(),
    docstore=docstore,
)
nodes = pipeline.run(documents=new_documents)

For production RAG with frequent updates, this saves significant engineering time.

Ecosystem and integrations

LangChain’s integration surface is wider — more vector stores, more document loaders, more LLM providers, more agent tools. If you need a niche loader (Confluence, Notion, Salesforce) or a specialized vector store (Milvus, Elasticsearch, Typesense), LangChain likely has a maintained integration.

LlamaIndex covers the major vector stores (Pinecone, Weaviate, Qdrant, Chroma, FAISS, pgvector, Milvus) and the common loaders (PDF, HTML, Notion, Slack, GitHub). Its LlamaHub registry is smaller but higher signal — integrations tend to be more feature-complete (e.g., the Pinecone integration supports namespaces, hybrid search, and metadata filtering natively).

Both frameworks support OpenAI-compatible endpoints. If you route through a gateway like n4n.ai for model fallback and usage metering, you pass the base URL and API key to either framework’s OpenAI client wrapper — no framework-specific code required.

Performance characteristics

Ingestion latency is dominated by embedding calls and vector store write throughput. Both frameworks batch embeddings by default (LangChain via embeddings.embed_documents, LlamaIndex via embed_model.get_text_embedding_batch). LlamaIndex’s IngestionPipeline parallelizes transformations across workers; LangChain leaves parallelism to you.

Query latency differs in retrieval composition. LangChain’s EnsembleRetriever runs retrievers sequentially unless you wrap them in RunnableParallel. LlamaIndex’s hybrid query mode executes dense and sparse retrieval in a single vector store call when the backend supports it (Pinecone, Weaviate, Qdrant), reducing round trips.

Reranking adds 50–200ms per query depending on model and candidate count. Both frameworks support async retrieval; LlamaIndex’s QueryEngine has aquery() and astream_query() built in, while LangChain uses ainvoke() on the retriever chain.

Comparison table

Which to choose

Choose LangChain when:

• You need a vector store or loader that LlamaIndex does not support (e.g., Elasticsearch, Typesense, niche SaaS connectors).
• Your RAG pipeline is simple — single vector store, top-k retrieval, no reranking, no hierarchical retrieval.
• You already have LangChain chains for agents, tool use, or chat memory and want consistency.
• You prefer explicit composition over opinionated abstractions and are comfortable wiring retrievers, rerankers, and synthesizers yourself.

Choose LlamaIndex when:

• You need semantic chunking, hierarchical retrieval, or parent-child node relationships out of the box.
• You want hybrid search and reranking without writing glue code.
• You have multi-tenant or versioned data requiring consistent metadata filtering across vector stores.
• You need an ingestion pipeline that handles deduplication, incremental updates, and caching reliably.
• You are building a knowledge graph or entity-aware RAG system (PropertyGraphIndex).
• You value citation-aware responses with MetadataReplacementPostProcessor for production audit trails.

Use both when:

• LangChain handles agent orchestration and tool calling while LlamaIndex powers the retrieval subsystem. Pass LlamaIndex retrievers as LangChain tools via create_retriever_tool.

The frameworks are not mutually exclusive. The pragmatic split is: LlamaIndex for the retrieval plane, LangChain for the reasoning plane.