Vector database observability: a practical checklist

A usable vector database observability checklist is not a grab-bag of dashboards; it is a disciplined set of signals that tell you when semantic search silently degrades. If you run embeddings in production, you need to watch the vector store with the same rigor you apply to your primary SQL database, because a 5% drop in recall rarely throws an error.

Measure query latency at percentiles, not averages

Averages hide the long tail that breaks user experience. Vector search latency is dominated by HNSW graph traversal and payload filtering; a single expensive filter can push p99 from 20 ms to 2 s while the mean stays flat.

Export a histogram with meaningful buckets and scrape it. If you are on Qdrant or Weaviate, wrap the client call:

from prometheus_client import Histogram
VECTOR_LATENCY = Histogram(
    "vector_query_latency_seconds",
    "End-to-end vector search latency",
    buckets=(0.01, 0.05, 0.1, 0.5, 1, 2, 5),
)

with VECTOR_LATENCY.time():
    results = client.query_points(collection, query_vector, limit=10)

Alert on p95 and p99 divergence, not on the average. When p99 climbs but p50 is stable, you have a hotspot—usually a large tenant, a missing index on a metadata field, or a degraded node.

Track recall and relevance against a golden set

Vector databases do not know if they returned the “right” documents. You must own a small, curated set of queries with known good IDs and compute recall offline or in a canary.

expected = golden_set[query_id]          # list of ground-truth doc ids
retrieved = [hit.id for hit in results]
recall = len(set(expected) & set(retrieved)) / len(expected)

Run this on a schedule against a shadow replica. A drop below your threshold (say 0.9) means the index parameters, embedding model, or data drift need attention. Do not trust the similarity score alone—cosine distance is not calibrated to business relevance.

Monitor index build time and segment counts

Index health is invisible until it isn’t. HNSW graphs in Qdrant, IVF in Milvus, or pgvector’s ivfflat all have background optimization. If segment counts grow unbounded or optimizer status is stuck, query latency degrades gradually.

info = client.get_collection("docs")
print(info.status, info.optimizer_status, info.vectors_count)

Graph the number of segments and the time since last optimization. A build that takes 10× longer than baseline indicates either a surge in writes or a node with insufficient RAM for the configured m/ef_construct parameters.

Expose per-collection cardinality and dimension drift

A collection that silently changes vector dimension will break queries or cause fallback to brute force. In pgvector you can catch this with a simple aggregate:

SELECT collection_id,
       count(*) AS rows,
       avg(array_length(embedding, 1)) AS dim
FROM documents
GROUP BY collection_id;

Track row count growth per collection and per tenant. Sudden cardinality spikes often precede out-of-memory kills during index rebuilds. Dimension drift usually means an embedding pipeline shipped a new model version without a migration—fail loud, not at query time.

Instrument embedding model version and dimension mismatches

The vector database is downstream of an embedding step. Stamp every inserted vector with the model name and dimension as payload or a side table. At query time, assert compatibility:

if len(query_vec) != stored_dim:
    raise ValueError(f"Dimension mismatch: {len(query_vec)} vs {stored_dim}")

If you rotate embedding models, run dual-write and dual-read before cutover. Observability here is about catching the mismatch before a user sees empty results, not after.

Capture slow queries and full trace context

A slow vector query is rarely isolated—it is part of a RAG request. Propagate trace context from the API layer into the vector call using OpenTelemetry:

from opentelemetry import trace
tracer = trace.get_tracer("vector")
with tracer.start_as_current_span("vector.search") as span:
    span.set_attribute("db.system", "qdrant")
    span.set_attribute("vector.limit", 10)
    span.set_attribute("vector.filter", str(filter_expr))

Sample 100% of traces in staging, 1–5% in production. The span should include the filter expression, effective ef value, and number of candidates visited. Without this, you cannot explain why one query was 50× slower than its twin.

Alert on disk, memory, and replication lag

Vector indexes are memory-hungry. HNSW keeps graph links in RAM; IVF keeps coarse centroids there. Set hard alerts on memory pressure and disk saturation:

- alert: VectorNodeMemoryHigh
  expr: node_memory_used_bytes / node_memory_total_bytes > 0.9
  for: 5m

If you run replicated clusters (Milvus, managed Pinecone), monitor lag between primary and replica. A replica lagging by more than a few seconds means newly inserted documents are invisible to a fraction of reads—a classic cause of “sometimes the answer is wrong” tickets.

Watch cost per query and per tenant

Most vector DBs charge by node hours or rows scanned. Even self-hosted clusters have a tangible EC2 bill. Tag every query with tenant ID and export a counter:

VECTOR_QUERIES_TOTAL.labels(tenant=tenant_id).inc()

Compute cost per tenant weekly. A single tenant doing unbounded limit=1000 scans will quietly dominate your bill. Set per-tenant quota on result size and concurrency at the gateway layer.

Validate backup and restore procedures

Observability includes knowing that recovery works. A vector index is not a append-only log; corruption during rebuild is possible. For pgvector, test restores regularly:

pg_dump -t documents -F c -f vec.dump prod_db
pg_restore -d restore_test vec.dump

For Milvus or Qdrant, use their snapshot tools and actually load the snapshot into a fresh instance. A backup you have not restored is a hypothesis, not a control.

Correlate vector DB metrics with upstream LLM calls

Retrieval is one stage in a pipeline. If you route generation through n4n.ai, its per-token metering and provider fallback will not mask a slow vector lookup—the overall request still stalls. Join the vector query span with the LLM completion span by trace ID.

When p99 of vector latency rises, check whether LLM time-to-first-token also rises. If they move together, the vector store is the bottleneck; if they decouple, look at model routing. This correlation turns a generic “slow responses” alert into a precise remediation.

Synthesis

The vector database observability checklist above is boring on purpose. None of these signals are exotic, but together they catch the failures that actually happen: silent recall loss, memory exhaustion during reindex, and tenant-level cost leaks. Ship the metrics first, the dashboards second, the alerts last.