Chunking strategies in LangChain: fixed size vs semantic

Retrieval-augmented generation lives or dies on how you slice source documents. The debate around langchain chunking fixed size vs semantic isn’t academic—it determines recall, latency, and your embedding bill before a single vector ever hits the index.

Fixed-size chunking: the default for a reason

LangChain’s RecursiveCharacterTextSplitter is the workhorse most pipelines copy from the docs and never revisit. You set a chunk_size in characters (or tokens if you wire a tokenizer) and an overlap, and the splitter walks a hierarchy of separators—paragraph, newline, sentence, space—to keep chunks as coherent as possible without exceeding the limit.

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=64,
    separators=["\n\n", "\n", ". ", " "],
)
chunks = splitter.split_text(long_doc)

The algorithm is purely local. It never calls a model, never needs a network, and produces identical output across runs. That determinism is a feature when you’re debugging why a specific paragraph disappeared from search. If you need token-accurate chunks, swap in TokenTextSplitter with a tiktoken encoder; the same fixed-size philosophy applies.

Where it breaks

Fixed-size splitting is blind to topic shifts. A 512-character window can cut a sentence mid-thought, or worse, separate a definition from its elaboration. Overlap mitigates but does not solve this; it just duplicates context at boundaries, inflating storage and embedding cost marginally. For code or tabular data where natural blocks exceed chunk_size, recursive splitting will mangle structure because it falls back to spaces and characters.

Semantic chunking: let embeddings decide boundaries

Semantic chunking inverts the control flow. Instead of imposing a size, you embed atomic units—usually sentences—and measure cosine similarity between adjacent units. When similarity drops below a threshold, you declare a boundary.

from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
semantic_splitter = SemanticChunker(
    embeddings,
    buffer_size=1,
    breakpoint_threshold_type="percentile",
    breakpoint_threshold_amount=95,
)
semantic_chunks = semantic_splitter.split_text(long_doc)

The buffer_size controls how many sentences to look back when deciding a breakpoint; the threshold type and amount tune sensitivity. Percentile mode treats the 95th percentile of dissimilarity as a cut. The splitter first sentences the document, embeds each, then walks the similarity sequence to find valleys.

Where it breaks

You now depend on embedding quality. A weak embedding model will fuse unrelated topics or shatter a single topic into fragments. Semantic chunking also produces variable-length chunks, which can blow past your vector store’s max token limit if you aren’t careful. Because it only merges adjacent sentences, it cannot reorganize disordered text—garbage in, garbage grouped.

Head-to-head dimensions

When evaluating langchain chunking fixed size vs semantic, stack them on the axes that affect production: capability, cost, latency, ergonomics, ecosystem, and hard limits.

Capabilities

Fixed-size: deterministic, order-preserving, trivially parallelizable, and can be made token-aware. Semantic: meaning-aware boundaries that group related sentences, still order-preserving because it only merges adjacent units. Semantic can surface a single coherent section from a wandering document; fixed-size cannot.

Price/cost model

Fixed-size costs zero inference tokens—pure CPU. Semantic requires one embedding vector per sentence (or per buffer window). At a million documents, that’s a real line item. If you route those embedding calls through a gateway like n4n.ai, you get per-token usage metering and automatic fallback when a provider is rate-limited or degraded, which turns a fragile batch job into a resilient pipeline. Semantic also imposes hidden cost: threshold changes force re-embedding if you cached nothing.

Latency/throughput

Fixed-size runs in microseconds on a single thread. Semantic makes O(n) synchronous embedding calls unless you batch them; even batched, you’re bounded by the embedding endpoint’s RPM. For offline ingestion of a 100k-doc corpus, semantic can be 10–100x slower in wall-clock time depending on network and batch size. Fixed-size scales linearly with document length and nothing else.

Ergonomics

Fixed-size exposes two integers and a separator list. Semantic exposes buffer_size, breakpoint_threshold_type (percentile, standard_deviation, interquartile, gradient), and breakpoint_threshold_amount. Misconfigure the threshold and you’ll get one giant chunk or one chunk per sentence. Fixed-size is easier to unit test; semantic needs a golden set of expected boundaries.

Ecosystem

Both are first-party LangChain. Fixed-size lives in langchain.text_splitter and is battle-tested. Semantic is in langchain_experimental (or the newer langchain.text_splitter in recent releases) and carries an experimental warning, but it’s stable enough for production if you pin versions. Both accept any LangChain Embeddings instance, so provider swaps are local changes.

Limits

Fixed-size struggles with documents where natural boundaries exceed chunk_size—code files, tables. Semantic struggles with low-resource languages if your embedding model isn’t multilingual, and it cannot enforce a hard max size without a post-hoc trim. Fixed-size overlap inflates chunk count; semantic can produce a chunk larger than your context window if a topic runs long.

Comparison table

Which to choose

The langchain chunking fixed size vs semantic decision ultimately boils down to your constraints, not ideology.

Use fixed-size when…

• You need offline or edge processing with no API dependency.
• Latency per ingest is critical (e.g., streaming ingestion of support tickets).
• Your documents are already well-structured (markdown with headings, code with line breaks) and a recursive splitter respects those separators.
• You’re prototyping and want zero external cost or moving parts.

Use semantic when…

• Recall on long, messy prose (legal contracts, research PDFs) is the product metric.
• You already pay for embeddings elsewhere in the pipeline, so marginal cost is low.
• Your embedding model is strong on the document language.
• You can batch embedding calls and tolerate slower backfills or precompute overnight.

Hybrid pattern

A common production setup: run semantic chunking to find topic boundaries, then apply a fixed-size splitter inside each semantic group to enforce a max token count. This captures meaning while staying within vector store limits.

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_experimental.text_splitter import SemanticChunker

def hybrid_split(doc, embeddings):
    semantic = SemanticChunker(
        embeddings,
        breakpoint_threshold_type="percentile",
        breakpoint_threshold_amount=90,
    )
    groups = semantic.split_text(doc)
    fixed = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=64)
    return [c for g in groups for c in fixed.split_text(g)]

That pattern adds the cost of embeddings but caps chunk size—best of both for RAG over unpredictable corpora.

Verdict by use case

• Static docs, low budget, strict latency: fixed-size.
• High-recall Q&A, messy sources: semantic (or hybrid).
• Multilingual corpus: semantic with multilingual embeddings, fixed-size as fallback.
• Real-time ingestion or air-gapped: fixed-size only.
• Regulated audit logs needing reproducible splits: fixed-size with pinned separator config.

Pick the splitter that matches your worst constraint, not the one that sounds smart. The rest is parameter tuning.