FAISS (Facebook AI Similarity Search) is an open-source library from Meta for efficient similarity search and clustering of dense vectors. It provides optimized implementations of approximate nearest neighbor (ANN) algorithms that scale to billions of vectors on a single GPU or across distributed CPU clusters. Unlike general-purpose vector databases, FAISS is a lower-level building block — you embed it directly in your application and manage persistence, sharding, and serving yourself.
How FAISS works
At its core, FAISS trades exact accuracy for massive speed and memory savings through approximate nearest neighbor search. The library organizes vectors into index structures that enable sub-linear query time. The most common index types fall into three categories:
Flat indexes store vectors verbatim and compute exact distances at query time. IndexFlatL2 and IndexFlatIP (inner product) are the baselines — accurate but O(N) per query. Use these when N < 100k or when you need ground truth for evaluation.
Inverted file (IVF) indexes partition the vector space using k-means clustering. At index time, vectors are assigned to their nearest centroids. At query time, you probe only the closest nprobe centroids, reducing the search space from N to roughly N/nlist × nprobe. The tradeoff: recall drops as you probe fewer lists.
Product quantization (PQ) and its variants compress vectors by splitting each vector into M sub-vectors and quantizing each subspace independently with a small codebook (typically 256 centroids per subspace). This reduces memory from 4×d bytes per vector to M bytes, enabling billion-scale indexes in RAM. IndexIVFPQ combines IVF coarse quantization with PQ fine quantization — the workhorse for large-scale deployments.
GPU acceleration is a first-class concern. FAISS implements brute-force, IVF, and PQ kernels for NVIDIA GPUs with automatic memory management. The same index API works on CPU and GPU; you migrate with index_cpu_to_gpu and index_gpu_to_cpu.
import faiss
import numpy as np
# 1M vectors, 128 dimensions
d = 128
nlist = 4096 # number of Voronoi cells
m = 16 # PQ sub-vectors (128/16 = 8 dims each)
nbits = 8 # 256 centroids per subspace
# Training data (typically a sample of your corpus)
train_vecs = np.random.random((100000, d)).astype('float32')
faiss.normalize_L2(train_vecs) # for inner product / cosine
# Build IVF+PQ index
quantizer = faiss.IndexFlatIP(d)
index = faiss.IndexIVFPQ(quantizer, d, nlist, m, nbits)
index.train(train_vecs)
# Add vectors in batches
batch_size = 100000
for i in range(0, 1_000_000, batch_size):
vecs = np.random.random((batch_size, d)).astype('float32')
faiss.normalize_L2(vecs)
index.add(vecs)
# Query
index.nprobe = 32
k = 10
query = np.random.random((1, d)).astype('float32')
faiss.normalize_L2(query)
distances, indices = index.search(query, k)Why FAISS matters for production systems
Vector search is the bottleneck in every RAG pipeline, recommendation engine, and duplicate detection system. FAISS matters because it makes ANN search fast enough to run in the request path without a separate service.
Latency profile: A well-tuned IVFPQ index on CPU serves 10k QPS with 10ms p99 on 10M vectors. On a single A100, the same workload hits 100k+ QPS. This eliminates the need for a dedicated vector database in many architectures — you can embed FAISS in your model server or API layer.
Memory efficiency: PQ compression lets you fit 1B vectors of dimension 768 in ~100 GB RAM (1 byte per sub-vector × 96 sub-vectors = 96 bytes/vector vs 3 KB for float32). This is the difference between fitting your index on one machine versus needing a distributed cluster.
No operational dependencies: FAISS is a C++ library with Python bindings. No separate process, no network hop, no configuration management. You deploy it like any other dependency. This simplicity is underrated — many teams adopt Pinecone or Weaviate because they assume FAISS requires a PhD to operate. It doesn’t.
Extensibility: The index factory pattern (faiss.index_factory) lets you compose index types declaratively: "IVF4096,PQ16" or "OPQ32_128,IVF4096,PQ16". You can also implement custom distance functions, pre-transforms (OPQ, random rotation), and on-disk indexes (IndexIVFPQ + OnDiskInvertedLists) for larger-than-RAM datasets.
Concrete example: RAG retrieval at scale
Consider a retrieval-augmented generation system serving 50M document chunks. Each chunk is embedded with a 1024-dim model (e.g., E5-large or BGE). You need top-20 recall > 0.95 at < 50ms p99.
# Production-grade index configuration
d = 1024
nlist = 32768 # ~1500 vectors per list at 50M scale
m = 32 # 32 bytes per vector (1024/32 = 32 dims/subvector)
nbits = 8
# OPQ (optimized product quantization) rotates space before PQ
# for better quantization error distribution
index = faiss.index_factory(d, "OPQ32_1024,IVF32768,PQ32")
# Train on 1M sample (2% of corpus)
train_sample = load_training_sample(1_000_000)
index.train(train_sample)
# Add in streaming batches, persist to disk
index = faiss.index_cpu_to_all_gpus(index) # multi-GPU if available
for batch in stream_corpus_batches(batch_size=50000):
index.add(batch)
if index.ntotal % 5_000_000 == 0:
faiss.write_index(index, f"checkpoint_{index.ntotal}.faiss")
# Final index
faiss.write_index(index, "production.faiss")
# Serving: nprobe tuned for recall target
index.nprobe = 128 # probes 128/32768 = 0.4% of listsTuning loop: Start with nprobe = nlist // 100. Measure recall@k against a held-out labeled set. Increase nprobe until recall target is met, then measure latency. If latency exceeds budget, increase nlist (more lists = smaller lists = faster per-list scan) and retrain. This is the only knob that matters for IVF indexes.
On-disk variant: For 500M+ vectors, keep the PQ codes in RAM and the inverted lists on NVMe:
# Build in RAM, then move inverted lists to disk
index = faiss.index_factory(d, "IVF65536,PQ32")
index.train(train_vecs)
index.add(all_vecs)
# Move inverted lists to memory-mapped file
ivf = faiss.extract_index_ivf(index)
ivf.replace_invlists(
faiss.OnDiskInvertedLists(ivf.nlist, ivf.code_size, "ivf_data.bin")
)
faiss.write_index(index, "ondisk.faiss")Common misconceptions
Misconception: FAISS is a vector database. FAISS has no persistence layer, no replication, no query parser, no access control, and no horizontal scaling protocol. It is an index library. You build the database around it: sharding, WAL, snapshotting, rolling updates, and a serving layer. Teams that treat FAISS as a drop-in database replacement end up rebuilding half of Milvus poorly.
Misconception: HNSW is always better than IVF. HNSW (Hierarchical Navigable Small World) graphs offer excellent recall/latency tradeoffs for million-scale indexes. But HNSW memory overhead is ~1.5-2× the raw vectors (graph edges + vectors), and it doesn’t compress. IVFPQ at billion scale uses 10-20× less memory. HNSW also degrades on high-dimensional vectors (d > 512) without dimensionality reduction. Choose HNSW for < 10M vectors where RAM is plentiful; choose IVFPQ for larger scale or constrained memory.
Misconception: You need GPU for FAISS to be fast.
CPU IVFPQ with nprobe=64 on a modern Xeon or Graviton3 achieves 5-10k QPS on 10M vectors. GPU shines when you need 50k+ QPS or when you’re doing brute-force search (exact k-NN) as a reranker. Most production workloads run fine on CPU; GPU is a cost optimization at high throughput, not a requirement.
Misconception: FAISS only does L2 distance.
IndexFlatIP computes inner product. Normalize your vectors (faiss.normalize_L2) and inner product equals cosine similarity. Most embedding models (BERT, E5, BGE, OpenAI) output vectors meant for cosine similarity. Use IndexFlatIP or IndexIVFIP — not L2 — unless your embeddings are explicitly trained for Euclidean distance.
Misconception: Training is optional.
Every IVF, PQ, HNSW, and OPQ index requires a training step on representative data. Skipping training (or training on garbage) produces indexes with catastrophic recall. The training sample should be at least 30× nlist vectors, drawn from the same distribution as your corpus. If your corpus shifts, retrain.
Misconception: Quantization always hurts recall.
PQ with m=16, nbits=8 on 768-dim vectors typically loses 1-3% recall@10 versus flat, while using 1/48th the memory. OPQ (optimized PQ) recovers most of that loss by learning a rotation that equalizes variance across subspaces. The recall/memory curve is favorable — don’t avoid quantization on principle.
When not to use FAISS
- You need filtered search (metadata + vector). FAISS has no native filter support. You either pre-filter (build separate indexes per tenant/category) or post-filter (retrieve 10× k, filter in application). Both are awkward at scale. Use a vector database with native filtering (Qdrant, Milvus, Weaviate) instead.
- You need ACID updates. FAISS supports
addandremove_ids, but removals are lazy (tombstones) and compaction requires rebuilding. High-churn workloads (millions of updates/day) need a database with LSM-tree or segment-based architecture. - Your team has no C++/Python ops capability. If you can’t debug a segfault in
faiss::IVFSearchParametersor tunenprobefrom latency/recall curves, a managed service will save you months.
Operational notes
Index versioning: Store the training data hash and index parameters alongside the index file. When embeddings change (model upgrade), you must rebuild. Automate this in your CI/CD.
Monitoring: Export index.ntotal, index.nprobe, query latency percentiles, and recall@k (via periodic evaluation sets). Alert on recall drift — it signals distribution shift or index corruption.
Warmup: On cold start, run a few thousand dummy queries to populate CPU caches and (if GPU) CUDA context. First-query latency is 10-100× higher.
Memory mapping: For on-disk indexes, use faiss.read_index("ondisk.faiss", faiss.IO_FLAG_MMAP) to avoid loading inverted lists into RAM until accessed. This lets you serve indexes larger than physical memory, provided your working set fits.
FAISS remains the foundation of large-scale vector search because it solves the hard algorithmic problems — quantization, clustering, GPU kernels — and leaves the systems problems to you. That’s the right boundary for a library. If you’re building a RAG system, a recommendation engine, or a deduplication pipeline at > 1M vectors, you should know FAISS well enough to decide whether to embed it or buy a database that wraps it.