Monitoring pgvector query latency in Postgres

pgvector query latency monitoring separates a demo RAG pipeline from a production one. When you store embeddings in Postgres and serve nearest-neighbor lookups to a live service, a silent plan change from index scan to sequential scan will spike tail latency without throwing an error. The steps below show how to instrument pgvector queries, capture execution statistics, and wire up alerts using native Postgres tooling and a small Python sampler.

Step 1: Enable the extensions and configure shared memory

You cannot monitor what the database does not track. Load pg_stat_statements into shared_preload_libraries so it collects per-statement timing from the moment the backend starts. The vector extension itself must be present for the <-> operator to exist.

CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

Edit postgresql.conf (or your managed Postgres parameter group) to preload the module and track all statements, not just top-level ones:

# postgresql.conf
shared_preload_libraries = 'pg_stat_statements'
pg_stat_statements.track = 'all'
pg_stat_statements.track_utility = on
pg_stat_statements.max = 10000

Restart Postgres. Verify both extensions are active:

SELECT extname FROM pg_extension
WHERE extname IN ('vector', 'pg_stat_statements');

If the query returns two rows, the foundation for pgvector query latency monitoring is in place.

Step 2: Baseline your vector queries with EXPLAIN ANALYZE

Before collecting aggregate stats, confirm the planner uses an index. Create a representative query with your real distance operator and run it with ANALYZE:

EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
SELECT id FROM products
ORDER BY embedding <-> '[0.12,0.04,0.88,0.33]'::vector
LIMIT 10;

Look for Index Scan using products_embedding_idx in the plan. If you see Seq Scan with an external sort, your ivfflat or hnsw index is not being used—often because the ORDER BY column is wrapped in a function or the index was built with wrong parameters. The Buffers line tells you how much memory and disk I/O the scan consumed; a cold index scan that hits 2000 buffers for a 10-row result is a latency bomb waiting to happen.

Run this on a warm cache and a cold cache (restart or DROP CLEAN BUFFERS) to record both best and worst cases.

Step 3: Capture aggregate latency with pg_stat_statements

pg_stat_statements normalizes queries and reports mean_exec_time, stddev_exec_time, and max_exec_time in milliseconds. Filter for the distance operator to isolate vector lookups:

SELECT
  query,
  calls,
  mean_exec_time,
  stddev_exec_time,
  max_exec_time
FROM pg_stat_statements
WHERE query ILIKE '%<->%'
ORDER BY mean_exec_time DESC;

This view is cumulative since the last reset. For windowed analysis, reset on a schedule (e.g., hourly from a cron job):

SELECT pg_stat_statements_reset();

A common mistake is assuming mean_exec_time reflects tail latency. It does not. A query with mean_exec_time of 3 ms and stddev_exec_time of 40 ms is periodically disastrous. Always pair mean with max and stddev when reviewing pgvector query latency monitoring dashboards.

Step 4: Run a high-resolution sampler

Aggregate views hide short-lived regressions. Deploy a sidecar script that issues a canned vector query every second and logs latency to a dedicated table. This gives you a time series independent of Postgres’ internal accounting.

First, the sink table:

CREATE TABLE vector_latency_samples (
  id bigserial PRIMARY KEY,
  sampled_at timestamptz DEFAULT now(),
  latency_ms double precision,
  hit_count int
);

Then a minimal Python loop using psycopg2:

import psycopg2, time

conn = psycopg2.connect("dbname=prod user=monitor password=secret")
cur = conn.cursor()
probe_vec = "[0.12,0.04,0.88,0.33]"  # representative centroid

while True:
    start = time.perf_counter()
    cur.execute(
        "SELECT id FROM products ORDER BY embedding <-> %s::vector LIMIT 10",
        (probe_vec,),
    )
    rows = cur.fetchall()
    elapsed = (time.perf_counter() - start) * 1000
    cur.execute(
        "INSERT INTO vector_latency_samples(latency_ms, hit_count) VALUES (%s, %s)",
        (elapsed, len(rows)),
    )
    conn.commit()
    time.sleep(1)

Run this under a dedicated low-privilege role. The script measures end-to-end round trip from the app tier, capturing network and connection overhead that pg_stat_statements excludes.

Step 5: Track index health and probe settings

pgvector’s ivfflat index requires a probes setting that trades recall for speed. Too few probes forces a broader scan; too many reads extra lists. Check and tune:

SHOW ivfflat.probes;
SET ivfflat.probes = 16;  -- session-level, or ALTER DATABASE ... SET

Monitor whether the index is actually being scanned versus the table:

SELECT
  indexrelname,
  idx_scan,
  pg_size_pretty(pg_relation_size(indexrelname::regclass)) AS size
FROM pg_stat_user_indexes
WHERE indexrelname LIKE '%embedding%';

If idx_scan is flat while seq_scan on the heap climbs, the planner has bailed on the index. This is the single most common cause of pgvector query latency monitoring dashboards lighting up after a Postgres upgrade or a VACUUM FULL that dropped the index silently.

For hnsw indexes (pgvector 0.5+), there is no probes knob, but you should still watch pg_stat_user_indexes and the index size, because HNSW build time and memory can affect write latency.

Step 6: Compute p95 and fire alerts

The sampler table enables proper percentile math. Calculate a rolling five-minute p95:

SELECT percentile_cont(0.95) WITHIN GROUP (ORDER BY latency_ms) AS p95_ms
FROM vector_latency_samples
WHERE sampled_at > now() - interval '5 minutes';

Wrap this in a check script that pages when p95 exceeds a threshold you derived from Step 2’s cold-cache baseline plus a 2x safety margin:

import psycopg2

conn = psycopg2.connect("dbname=prod user=monitor")
cur = conn.cursor()
cur.execute("""
  SELECT percentile_cont(0.95) WITHIN GROUP (ORDER BY latency_ms)
  FROM vector_latency_samples
  WHERE sampled_at > now() - interval '5 minutes';
""")
p95 = cur.fetchone()[0] or 0
if p95 > 50.0:  # ms, adjust to your SLO
    print(f"ALERT: vector p95 {p95:.1f}ms exceeds 50ms")
    # trigger webhook, raise metric, etc.

Do not alert on a single slow sample; use the percentile to avoid noise from garbage collection pauses in the client.

Step 7: Validate end-to-end with a load test

Monitoring is only trustworthy if it survives real concurrency. Generate synthetic load with pgbench using a custom script that issues the same vector query:

cat > vector_query.sql <<'EOF'
\set emb '[0.12,0.04,0.88,0.33]'
SELECT id FROM products ORDER BY embedding <-> :emb::vector LIMIT 10;
EOF

pgbench -f vector_query.sql -c 8 -T 60 -h localhost -U postgres prod

While this runs, watch vector_latency_samples ingest rows and confirm the p95 query from Step 6 reflects the elevated load. Success criteria:

1. The sampler table shows latency increasing under -c 8 but staying under your alert threshold (or crossing it deliberately if you set the threshold low for the test).
2. pg_stat_statements shows calls incrementing on the <-> query and mean_exec_time tracking the load-test window after a reset.
3. The EXPLAIN ANALYZE plan during load still shows Index Scan, not Seq Scan.

If all three hold, your pgvector query latency monitoring pipeline is operational. If the index drops under concurrency, raise ivfflat.probes or consider HNSW, then re-run the load test to confirm the plan stabilizes.

One last note: keep the sampler query representative of production filters. If your real queries add WHERE category = $1, include that column in the index as a composite or accept that monitoring a bare <-> scan will understate latency. Accurate pgvector query latency monitoring demands fidelity to the hot path, not a sanitized approximation.