Chunking bugs in RAG pipelines silently degrade retrieval quality. Testing LlamaIndex node parsers before they hit production saves you from mismatched context windows, lost metadata, and broken parent-child links. This how-to gives you a reproducible pytest workflow to catch those defects early and keep your splitters honest.
Step 1: Pin versions and isolate the parser
Start by pinning llama-index-core and pytest in a clean virtualenv. Node parser behavior changes across minor releases—especially SentenceSplitter token counting and separator handling—so a floating version hides regressions behind coincidental passes.
python -m venv .venv && source .venv/bin/activate
pip install "llama-index-core==0.10.43" pytest==8.2.0Import the parser from llama_index.core.node_parser. Avoid the legacy llama_index meta-package; the core split is stable and reduces dependency bloat.
from llama_index.core.node_parser import SentenceSplitter
from llama_index.core import DocumentVerification: pip freeze | grep llama-index-core shows the exact pin, and python -c "import llama_index.core" exits 0. If you see a different version, your environment is lying to you.
Step 2: Build deterministic fixtures
Random text gives flaky tests. Construct Document objects with known sentence counts, markdown headers, and fenced code blocks. A parser that splits inside a code fence or merges across headers is a classic bug that only shows under structured input.
import pytest
@pytest.fixture
def md_doc():
text = """# Title
First sentence. Second sentence. Third sentence.
```python
def foo():
return 1
```
Fourth sentence. Fifth sentence."""
return Document(text=text, metadata={"source": "fixture"})
@pytest.fixture
def plain_doc():
return Document(
text=" ".join(f"Sentence {i}." for i in range(20)),
metadata={"source": "plain"},
)Use these fixtures in every test. If a test mutates a doc, call md_doc.copy() to prevent cross-test leakage. Deterministic input is the only way to assert exact node counts later.
Step 3: Configure explicit split constraints
Instantiate SentenceSplitter with hard limits. The default chunk_size=1024 tokens hides bugs that only appear at tighter bounds used by smaller embedding models.
def make_parser(chunk_size=50, chunk_overlap=10):
return SentenceSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
separator=" ",
paragraph_separator="\n\n",
)Tokenization depends on the tokenizer. LlamaIndex uses tiktoken for OpenAI models by default; pass a tokenizer callable if you target a local model. Never rely on character count as a proxy for tokens in assertions—CJK text will betray you.
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
parser = SentenceSplitter(chunk_size=50, tokenizer=enc.encode)Step 4: Assert node count and size invariants
Write a test that parses the fixture and checks every node respects the cap. Overlap should appear between consecutive nodes but never exceed the limit.
def test_chunk_size_invariants(md_doc):
parser = make_parser(50, 10)
nodes = parser.get_nodes_from_documents([md_doc])
assert len(nodes) >= 1
for node in nodes:
toks = len(parser.tokenizer(node.get_content()))
assert toks <= 50, f"Node exceeded cap: {toks}"
if len(nodes) > 1:
tail = nodes[0].get_content().split()[-10:]
head = nodes[1].get_content().split()[:10]
assert any(t in head for t in tail), "Expected overlap missing"Run pytest -q tests/test_parsers.py::test_chunk_size_invariants. Green means the parser respects your contract. If it fails, print node.get_content() to see where the splitter went rogue.
Step 5: Verify metadata and relationships
A chunking bug often drops metadata or severs parent_id links. LlamaIndex nodes carry metadata and relationships. Test both.
def test_metadata_propagation(plain_doc):
parser = make_parser(50, 10)
nodes = parser.get_nodes_from_documents([plain_doc])
for node in nodes:
assert node.metadata["source"] == "plain"
assert node.ref_doc_id == plain_doc.doc_id
assert node.relationships.get("parent") is not None or node.ref_doc_id == plain_doc.doc_idIf you use HierarchicalNodeParser, assert that child nodes point to parent via relationships["parent"]. Broken links break recursive retrieval and are painful to debug in production.
from llama_index.core.node_parser import HierarchicalNodeParser
def test_hierarchical_links(md_doc):
parser = HierarchicalNodeParser(chunk_sizes=[50, 100])
nodes = parser.get_nodes_from_documents([md_doc])
child = [n for n in nodes if n.relationships.get("parent")][0]
assert child.relationships["parent"].node_id in {n.node_id for n in nodes}Step 6: Attack edge cases
Real corpora contain empty docs, giant single lines, and Unicode with no spaces. Add these fixtures and assert the parser survives.
@pytest.fixture
def edge_docs():
return [
Document(text=""),
Document(text="あ" * 2000), # CJK no spaces
Document(text="<html><body>no sentence breaks</body></html>"),
Document(text=" \n\t "), # whitespace only
]
def test_edge_cases(edge_docs):
parser = make_parser(50, 10)
for doc in edge_docs:
nodes = parser.get_nodes_from_documents([doc])
assert all(isinstance(n.get_content(), str) for n in nodes)
assert all(len(parser.tokenizer(n.get_content())) <= 50 for n in nodes)A parser that raises on empty input or explodes CJK into single characters fails here. Fix by setting paragraph_separator or a custom tokenizer that respects word boundaries.
Step 7: Property-based testing for split monotonicity
Use hypothesis to fuzz text length and assert no node exceeds cap and that concatenation reconstructs the source (minus separators). This catches off-by-one token bugs that hand-written fixtures miss.
pip install hypothesis==6.102.0from hypothesis import given, strategies as st
@given(st.text(min_size=0, max_size=5000))
def test_fuzz_chunk_cap(text):
doc = Document(text=text)
parser = make_parser(50, 10)
nodes = parser.get_nodes_from_documents([doc])
for n in nodes:
assert len(parser.tokenizer(n.get_content())) <= 50Run with pytest --hypothesis-profile=ci. If a generated string breaks the cap, Hypothesis shrinks it to a minimal repro you can paste into a regression test.
Step 8: Snapshot regression on chunk boundaries
When you upgrade LlamaIndex, chunk boundaries may shift subtly. Store node texts as JSON and compare against a committed baseline.
import json, os
def test_snapshot(md_doc, tmp_path):
parser = make_parser(50, 10)
nodes = parser.get_nodes_from_documents([md_doc])
snapshot = [n.get_content() for n in nodes]
os.makedirs("snapshots", exist_ok=True)
path = "snapshots/md.json"
if os.path.exists(path):
with open(path) as f:
assert snapshot == json.load(f), "Chunk boundaries drifted"
else:
with open(path, "w") as f:
json.dump(snapshot, f)Run once to generate, commit snapshots/md.json. CI fails on unexpected drift, forcing a human to review the new chunks.
Step 9: Wire into CI and confirm success
Add a pytest job to GitHub Actions or equivalent. Keep it strict.
jobs:
test-parsers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install -r requirements.txt pytest hypothesis
- name: Test parsers
run: pytest tests/test_parsers.py --strict-markers -qSuccess criterion: exit code 0, and the test file covers size, overlap, metadata, edge cases, and snapshot. If a test fails, the diff shows exactly which invariant broke—size, metadata, or overlap. Testing LlamaIndex node parsers this way turns a black-box splitter into a guarded component you can upgrade with confidence.
Step 10: Debug a real failure
Suppose test_chunk_size_invariants fails with Node exceeded cap: 54. Print the node content and you’ll likely see the splitter counted a code fence as one token blob but emitted it whole. The fix is to pre-split on ``` boundaries or use MarkdownNodeParser before SentenceSplitter. Write a new fixture with that exact content, confirm the fix, and the bug stays dead.
Following these steps gives you a test suite that makes chunking bugs loud instead of silent.