Building semantic product search llamaindex-style means turning a static catalog into a retrieval system that understands intent, not just keywords. This hands-on tutorial takes a small product JSON file from disk to a queryable vector index, then layers in metadata filters and persistence. You’ll run every snippet locally without spending on API calls.
Prerequisites
- Python 3.10 or newer
- Basic familiarity with Python and pip
- No external API keys required for the core build; we use local HuggingFace embeddings
- Install the minimal packages:
pip install llama-index-core llama-index-embeddings-huggingfaceIf you later swap to a hosted embedding model, add llama-index-embeddings-openai.
The product catalog
We’ll use a tiny catalog of six items. Save as products.json:
[
{"id": "p1", "name": "Trail Runner Pro", "description": "Lightweight breathable running shoes for off-road trails", "category": "shoes", "price": 89.99},
{"id": "p2", "name": "City Jogger", "description": "Cushioned road running shoes for daily commute", "category": "shoes", "price": 49.99},
{"id": "p3", "name": "Summer Floral Dress", "description": "Airy cotton dress with floral pattern for hot days", "category": "apparel", "price": 34.99},
{"id": "p4", "name": "Insulated Bottle", "description": "Keeps drinks cold 24h, fits bike cage", "category": "accessories", "price": 19.99},
{"id": "p5", "name": "Waterproof Hiking Jacket", "description": "Seam-sealed shell for heavy rain in mountains", "category": "apparel", "price": 129.99},
{"id": "p6", "name": "Yoga Mat", "description": "Non-slip biodegradable mat for home practice", "category": "accessories", "price": 29.99}
]In a real system this file is replaced by a database dump or a stream from your PIM. The shape matters: keep a human-readable text field and push structured attributes into metadata.
Load and embed
LlamaIndex treats each product as a Document. We concatenate the name and description into the document text and attach category and price as metadata so we can filter later. The embedding model runs locally via HuggingFaceEmbedding—no network calls, no API keys.
import json
from llama_index.core import Document, VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
with open("products.json") as f:
products = json.load(f)
docs = [
Document(
text=f"{p['name']}: {p['description']}",
metadata={"id": p["id"], "category": p["category"], "price": p["price"]},
)
for p in products
]
index = VectorStoreIndex.from_documents(docs)The bge-small-en-v1.5 model is roughly 130MB and gives decent English embeddings for dev work. It indexes the concatenated text as a single vector; we are not chunking because each product is short. For longer product blurbs, set chunk_size=256 on the loader or use a SentenceSplitter node parser.
Query the index
For semantic product search llamaindex retriever is enough—we want the matching products, not a synthesized paragraph. Use as_retriever and inspect the returned nodes. The default in-memory vector store returns nodes ranked by cosine similarity.
retriever = index.as_retriever(similarity_top_k=3)
results = retriever.retrieve("something for running on pavement")
for r in results:
print(r.node.metadata["id"], "|", r.node.text, f"| score={r.score:.3f}")Expected output (scores vary slightly by model version):
p2 | City Jogger: Cushioned road running shoes for daily commute | score=0.712
p1 | Trail Runner Pro: Lightweight breathable running shoes for off-road trails | score=0.658
p4 | Insulated Bottle: Keeps drinks cold 24h, fits bike cage | score=0.402The retriever ranked the road shoe first, then the trail shoe, then an unrelated bottle. That is correct semantic behavior: “running on pavement” maps to “road running shoes” and “daily commute”. The bottle scores far lower but still surfaces because the vector space is small.
Understand the scores
LlamaIndex returns score as the similarity metric used by the vector store. For the default SimpleVectorStore with cosine similarity, higher is better and values land between -1 and 1 after normalization. Don’t treat the absolute number as a probability. If you need a hard cutoff, wrap the retriever with a SimilarityPostprocessor:
from llama_index.core.postprocessor import SimilarityPostprocessor
retriever = index.as_retriever(
similarity_top_k=5,
node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.5)],
)This drops the bottle in the earlier example, leaving only the two shoes.
Add metadata filters
A real store filters by category or price. LlamaIndex passes MetadataFilters to the underlying vector store. The default in-memory store supports exact matches and numeric ranges.
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter, RangeFilter
filters = MetadataFilters(
filters=[
ExactMatchFilter(key="category", value="shoes"),
RangeFilter(key="price", gt=0, lt=60),
]
)
filtered_retriever = index.as_retriever(similarity_top_k=5, filters=filters)
shoe_results = filtered_retriever.retrieve("comfortable shoes for the city")
for r in shoe_results:
print(r.node.metadata["id"], r.node.metadata["price"], r.node.text)Expected output:
p2 49.99 City Jogger: Cushioned road running shoes for daily commuteOnly the sub-$60 shoe appears. The trail runner at $89.99 is excluded by the range filter before ranking happens. Note: not every vector database supports RangeFilter with the same semantics—pgvector does via metadata JSONB, but some managed stores require pre-defined scalar fields.
Persist the index
Re-embedding on every boot wastes compute. Persist the index to disk and reload it in a separate process.
index.storage_context.persist(persist_dir="./storage")
# Later, in a new process:
from llama_index.core import StorageContext, load_index_from_storage
ctx = StorageContext.from_defaults(persist_dir="./storage")
reloaded = load_index_from_storage(ctx)The serialized artifacts include the vector store, document store, and index metadata. On reload, queries run against the frozen embeddings. If your catalog changes, call index.refresh_ref_docs(docs) or rebuild—LlamaIndex does not auto-sync external JSON.
Production swap: embeddings and LLMs
Local models are great for dev, but production catalogs need stronger embeddings and possibly an LLM to rewrite ambiguous queries. LlamaIndex speaks the OpenAI client protocol, so any OpenAI-compatible endpoint drops in without code changes to your indexing logic.
If you route through an inference gateway such as n4n.ai, the same OpenAI embedding client works and you get automatic fallback when a provider is rate-limited or degraded, plus per-token metering. Configure it by pointing api_base at the gateway and setting your key.
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI
Settings.embed_model = OpenAIEmbedding(
model="text-embedding-3-small",
api_base="https://api.n4n.ai/v1", # OpenAI-compatible
api_key="YOUR_KEY",
)
Settings.llm = OpenAI(
model="gpt-4o-mini",
api_base="https://api.n4n.ai/v1",
api_key="YOUR_KEY",
)After this swap, the VectorStoreIndex.from_documents call embeds against the remote model; the rest of the semantic product search llamaindex code stays identical. The gateway forwards provider cache-control hints, so repeated catalog embeds with stable IDs can hit cache and cut cost.
Scaling beyond six items
The in-memory store is fine for a demo but falls over at 100k SKUs. Swap StorageContext for a persistent vector database:
from llama_index.vector_stores.milvus import MilvusVectorStore
vector_store = MilvusVectorStore(uri="http://localhost:19530", collection_name="products")
storage_ctx = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(docs, storage_context=storage_ctx)The retriever and filter APIs do not change. You only pay for the vector DB query latency instead of linear scans.
Recap
You now have a runnable pipeline: load JSON, embed locally with LlamaIndex, retrieve semantically, filter by metadata, persist to disk, and optionally move to a managed embedding endpoint. The core retrieval logic—the part that makes semantic product search llamaindex-powered—is unchanged whether you run on a laptop or behind a gateway serving 240+ models. From here, add a thin API layer (FastAPI) and a front-end search box, and you have a production-grade storefront search.