Dot product vs cosine similarity for embeddings

You’re building a retrieval system. The embedding model spits out vectors. Now you need to score query against document. The two standard choices are dot product and cosine similarity. They’re mathematically related — cosine is dot product on normalized vectors — but the operational differences matter: index build time, query latency, score interpretability, and what happens when your vectors aren’t unit length. This post breaks down the concrete tradeoffs so you can pick the right one without guessing.

The math in three lines

Dot product: score = sum(q_i * d_i) for i in dimensions.

Cosine similarity: score = dot(q, d) / (||q|| * ||d||).

If every vector in your index and every query vector is L2-normalized to unit length, the denominator is always 1.0 and the two scores are identical. The differences emerge when vectors aren’t normalized — either by design or by accident.

What normalization actually buys you

Most embedding models (OpenAI text-embedding-3-large, Cohere embed-english-v3, BGE, E5) output normalized vectors by default. But “default” isn’t a guarantee. Fine-tuned models, distilled models, or vectors passed through a projection layer often lose unit norm. If you’re concatenating embeddings, averaging them, or applying learned linear transforms, you almost certainly have non-unit vectors.

When vectors aren’t normalized, dot product conflates direction (semantic alignment) with magnitude (vector length). A long vector that points vaguely in the right direction can outscore a short vector that points exactly right. Cosine similarity discards magnitude entirely — only angle matters.

import numpy as np

q = np.array([1.0, 0.0])          # unit query
d1 = np.array([0.9, 0.1])         # short, aligned
d2 = np.array([10.0, 1.0])        # long, slightly off

print("dot(q, d1):", np.dot(q, d1))    # 0.9
print("dot(q, d2):", np.dot(q, d2))    # 10.0  <-- wins on dot product
print("cos(q, d1):", np.dot(q, d1) / (np.linalg.norm(q) * np.linalg.norm(d1)))  # 0.99
print("cos(q, d2):", np.dot(q, d2) / (np.linalg.norm(q) * np.linalg.norm(d2)))  # 0.995

In this toy example the ranking flips. In production, magnitude often correlates with document length, token count, or training artifacts — rarely with relevance.

Indexing and query-time cost

Dot product with pre-normalized vectors

If you control the index build and can guarantee unit vectors, dot product is faster. No division, no square roots at query time. Most vector databases (FAISS, HNSWlib, pgvector, Milvus, Weaviate, Pinecone) implement inner product search as a first-class metric. The index stores raw vectors; the search kernel computes dot(q, d) directly.

# FAISS example: inner product index on pre-normalized vectors
import faiss

dim = 1536
index = faiss.IndexFlatIP(dim)  # inner product = dot product
# assumes vectors are already L2-normalized
index.add(doc_vectors)
scores, ids = index.search(query_vectors, k=10)

Cosine similarity or non-normalized vectors

If vectors aren’t normalized, you have two options:

1. Normalize at index build time — store v / ||v|| in the index, then use dot product at query time. One-time cost at ingest. Query stays fast.
2. Compute cosine at query time — store raw vectors, compute dot(q, d) / (||q|| * ||d||) per candidate. This adds a division and two norms per scored vector. On HNSW or IVF indexes that score thousands of candidates per query, the overhead is measurable.

# FAISS: cosine via normalized vectors (recommended)
index = faiss.IndexFlatIP(dim)
faiss.normalize_L2(doc_vectors)   # in-place, once at build
index.add(doc_vectors)
faiss.normalize_L2(query_vectors) # per query batch
scores, ids = index.search(query_vectors, k=10)

Some databases (pgvector, Elasticsearch) expose cosine as a native metric and handle normalization internally. Check your engine’s docs — the implementation may be more optimized than a manual divide.

Score interpretability and thresholding

Cosine similarity returns scores in [-1, 1]. Positive means aligned, negative means opposed, zero means orthogonal. This makes thresholding intuitive: cos > 0.8 is a strong match, cos > 0.3 is weak. You can explain the threshold to a product manager.

Dot product on non-normalized vectors has unbounded range. A score of 12.4 means nothing without context — you need to know the typical magnitude distribution of your index. Thresholds become dataset-dependent and drift when you re-embed with a new model.

Even with normalized vectors, dot product equals cosine, so the range is [-1, 1]. But the default in many libraries is to return raw dot product without documenting the normalization assumption. If a future model change drops normalization, your thresholds silently break.

Training objectives and model compatibility

Contrastive learning objectives (InfoNCE, triplet loss, multiple negatives ranking loss) typically optimize for cosine similarity. The loss computes exp(cos(q, d+) / tau) / sum(exp(cos(q, d) / tau)). The model learns to arrange vectors on the hypersphere. At inference, cosine is the natural metric.

Some retrieval architectures (e.g., two-tower models with dot-product scoring heads, or late-interaction models like ColBERT) are trained with raw dot product. If you’re using a model from a paper that reports dot-product scores, match the training metric.

Practical rule: use the metric the model was trained with. If the model card doesn’t say, assume cosine — it’s the safer default for general-purpose embedding models.

Quantization and compression interactions

Product quantization (PQ), scalar quantization (SQ), and binary quantization all assume a distance metric. FAISS’s IndexPQ with METRIC_INNER_PRODUCT expects pre-normalized vectors. If you feed non-normalized vectors to a PQ index built for inner product, the quantization error distorts both direction and magnitude, and recall tanks.

Cosine similarity with quantization is trickier. You can:

• Normalize → quantize → search with dot product (standard approach)
• Store raw vectors → quantize → reconstruct → compute cosine at query time (slower, more memory)

Binary quantization (1-bit per dimension) works well with cosine if you normalize first. The Hamming distance between binary codes approximates angular distance. But dot product on unnormalized binary codes is meaningless.

Hybrid search and score fusion

In hybrid retrieval (BM25 + vector), you need to fuse scores from different scales. BM25 scores are roughly [0, 20+]. Cosine is [-1, 1]. Dot product on normalized vectors is also [-1, 1]. Dot product on non-normalized vectors could be [0, 1000].

Normalization makes fusion straightforward: min-max scale each signal to [0, 1], then weighted sum. With unbounded dot product, you need robust scaling (percentile clipping, log transform) or learned fusion — more moving parts, more failure modes.

def minmax_scale(scores, lo=None, hi=None):
    lo = lo or np.percentile(scores, 1)
    hi = hi or np.percentile(scores, 99)
    return np.clip((scores - lo) / (hi - lo), 0, 1)

bm25_norm = minmax_scale(bm25_scores)
vec_norm = minmax_scale(cosine_scores)  # or dot scores if normalized
fused = 0.6 * vec_norm + 0.4 * bm25_norm

Comparison table

When vectors aren’t normalized: the silent failure mode

The most common production bug: you switch embedding models, the new model doesn’t normalize, your index still uses dot product, and relevance degrades silently. No errors, no alerts — just worse recall.

Defensive patterns:

# 1. Assert at index build
def build_index(vectors, metric="cosine"):
    if metric == "cosine":
        norms = np.linalg.norm(vectors, axis=1)
        assert np.allclose(norms, 1.0, atol=1e-3), "Vectors must be normalized for cosine"
    elif metric == "dot":
        # optional: warn if not normalized
        norms = np.linalg.norm(vectors, axis=1)
        if not np.allclose(norms, 1.0, atol=1e-3):
            print(f"Warning: vector norms range [{norms.min():.3f}, {norms.max():.3f}]")
    # ... build index

# 2. Normalize defensively at query time (cheap)
def search(index, query_vectors, metric="cosine"):
    if metric in ("cosine", "dot"):
        faiss.normalize_L2(query_vectors)  # idempotent if already normalized
    return index.search(query_vectors, k=10)

The faiss.normalize_L2 call is ~50ns per vector on modern CPUs — negligible compared to network round trips or encoder inference.

Which to choose

Use cosine similarity (or dot product on normalized vectors) when:

• You’re using general-purpose embedding models (OpenAI, Cohere, BGE, E5, Jina, Nomic).
• You want interpretable scores and stable thresholds.
• You’re doing hybrid search with BM25 or other bounded signals.
• You’re using product quantization or binary quantization.
• You don’t control the embedding pipeline end-to-end (model updates, third-party APIs).

Use raw dot product (non-normalized) when:

• You’re using a model explicitly trained with dot-product scoring (some two-tower architectures, certain late-interaction heads).
• Magnitude carries signal — e.g., you’ve engineered vector length to represent confidence, document authority, or recency.
• You’re in a closed loop where you control training, indexing, and querying, and you’ve validated that magnitude helps.

Use dot product on pre-normalized vectors when:

• You’ve verified your vectors are unit length (model default + no post-processing that breaks it).
• You want the absolute fastest query path and your vector database optimizes inner product better than cosine.
• You’re building a high-throughput, latency-sensitive path and have tests that catch normalization drift.

One more thing: provider routing and fallback

If your retrieval pipeline calls an embedding API (OpenAI, Cohere, etc.) and that provider degrades or rate-limits, your vector norms can shift if the fallback model behaves differently. A gateway that handles automatic fallback across 240+ models while preserving your routing directives — and forwards provider cache-control hints so you don’t re-embed unchanged inputs — keeps the embedding distribution stable. That stability matters more than the dot-vs-cosine choice: a consistent wrong metric beats an inconsistent right one.

Summary

Default to cosine similarity. It’s robust, interpretable, and matches how most embedding models are trained. If you need the last 1-2% of query latency and you own the full stack — model, index, query — normalize once at build time and use dot product. Never use raw dot product on non-normalized vectors unless you have a specific, validated reason. And whichever you pick, assert your normalization assumptions in code so a model upgrade doesn’t silently break retrieval.