How cosine similarity measures semantic closeness

Cosine similarity measures semantic closeness by computing the cosine of the angle between two embedding vectors, giving you a score from -1 to 1 that reflects how aligned their directions are in high-dimensional space. Unlike Euclidean distance, it ignores magnitude and focuses purely on orientation — which is exactly what matters when comparing semantic meaning encoded in normalized embeddings. This guide walks through the math, the implementation, and the practical decisions you’ll face when putting it in production.

Step 1: Understand the geometric intuition

Embeddings from models like BERT, OpenAI’s text-embedding-3-large, or open-weight alternatives live on a hypersphere when properly normalized. Two sentences with similar meaning point in roughly the same direction; unrelated sentences point in orthogonal or opposite directions.

The cosine of the angle between vectors a and b is:

cos(θ) = (a · b) / (||a|| ||b||)

When both vectors are unit length (L2 normalized), this simplifies to a dot product:

cos(θ) = a · b

This is why every major embedding API returns normalized vectors — or expects you to normalize them. If you skip normalization, magnitude differences (often caused by token count or model quirks) will dominate the score and destroy semantic signal.

Key insight: Cosine similarity is not a distance metric. It’s a similarity score. Higher = more similar. If you need a proper distance for algorithms like HNSW or k-means, convert it: distance = 1 - cosine_similarity.

Step 2: Prepare your embeddings

Before computing anything, you need embeddings. Here’s a minimal pipeline using sentence-transformers — a reliable, CPU-friendly choice for local development.

# embed.py
from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

def embed(texts: list[str]) -> np.ndarray:
    """Return L2-normalized embeddings as float32."""
    vecs = model.encode(texts, convert_to_numpy=True, normalize_embeddings=True)
    return vecs.astype(np.float32)

if __name__ == "__main__":
    sentences = [
        "The cat sat on the mat",
        "A kitten rested on the rug",
        "The stock market crashed today",
        "Equities plummeted this morning",
    ]
    embeddings = embed(sentences)
    print(f"Shape: {embeddings.shape}")  # (4, 384)
    print(f"Norms: {np.linalg.norm(embeddings, axis=1)}")  # all ~1.0

Run it:

pip install sentence-transformers numpy
python embed.py

Verify success: norms should print as [1. 1. 1. 1.] (or within floating-point epsilon). If not, your embeddings aren’t normalized — fix that before proceeding.

Step 3: Compute cosine similarity

With normalized vectors, cosine similarity is a dot product. Here are three ways to compute it, from explicit to production-ready.

Option A: Explicit loop (for understanding)

def cosine_similarity_loop(a: np.ndarray, b: np.ndarray) -> float:
    """Single pair, explicit formula."""
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

Option B: Vectorized pairwise (for batches)

def cosine_similarity_matrix(embeddings: np.ndarray) -> np.ndarray:
    """Return (n, n) similarity matrix for normalized embeddings."""
    # embeddings shape: (n, d), already L2 normalized
    return embeddings @ embeddings.T  # dot product = cosine similarity

Option C: Scikit-learn (battle-tested, handles non-normalized input)

from sklearn.metrics.pairwise import cosine_similarity

def cosine_similarity_sklearn(a: np.ndarray, b: np.ndarray | None = None) -> np.ndarray:
    """Works whether or not inputs are normalized."""
    return cosine_similarity(a, b)

Production tip: If you control the embedding pipeline, normalize once at write time and use Option B. It’s a single BLAS call, no overhead. If you consume embeddings from multiple sources (some normalized, some not), use Option C — it’s safer and the overhead is negligible for typical batch sizes.

Step 4: Interpret the scores

Cosine similarity ranges from -1 to 1. In practice with modern embeddings:

Thresholds depend on your model, domain, and task. Calibrate on your data. Here’s a quick calibration script:

# calibrate.py
import numpy as np
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

pairs = [
    ("cat on mat", "kitten on rug", "paraphrase"),
    ("cat on mat", "dog on log", "same_structure"),
    ("cat on mat", "stock market crash", "unrelated"),
    ("buy low sell high", "purchase cheap sell dear", "paraphrase_finance"),
    ("buy low sell high", "market bull run", "related_finance"),
]

texts = [t for pair in pairs for t in pair[:2]]
embeddings = model.encode(texts, normalize_embeddings=True)

for i, (a, b, label) in enumerate(pairs):
    sim = cosine_similarity([embeddings[2*i]], [embeddings[2*i + 1]])[0, 0]
    print(f"{label:20s}  {sim:.4f}")

Typical output on MiniLM-L6-v2:

paraphrase              0.8231
same_structure          0.6812
unrelated               0.1945
paraphrase_finance      0.7918
related_finance         0.6123

Use this to pick a threshold for your use case. Semantic search often works well around 0.65–0.75; deduplication needs 0.85+.

Step 5: Handle edge cases and normalization pitfalls

Non-normalized embeddings

Some APIs (older OpenAI models, some open-weight checkpoints) return non-normalized vectors. Always verify:

def ensure_normalized(embeddings: np.ndarray, eps: float = 1e-8) -> np.ndarray:
    """L2 normalize rows, guarding against zero vectors."""
    norms = np.linalg.norm(embeddings, axis=1, keepdims=True)
    norms = np.maximum(norms, eps)
    return embeddings / norms

Zero vectors

A zero vector has no direction. Cosine similarity is undefined. The guard above prevents NaNs, but you should log or drop zero vectors upstream — they indicate a failed embedding call.

Dimensionality mismatch

def safe_cosine_similarity(a: np.ndarray, b: np.ndarray) -> np.ndarray:
    assert a.shape[1] == b.shape[1], f"Dimension mismatch: {a.shape[1]} vs {b.shape[1]}"
    a = ensure_normalized(a)
    b = ensure_normalized(b)
    return a @ b.T

Numerical precision

For very large batches, float32 dot products can accumulate error. If you need high precision (e.g., for exact top-k ranking), compute in float64:

sim = (a.astype(np.float64) @ b.astype(np.float64).T).astype(np.float32)

The cast back to float32 keeps downstream memory usage low.

Step 6: Scale to production

Real systems need to query millions of vectors. You have three architectural paths:

Path A: Exact search with NumPy (up to ~100k vectors)

# exact_search.py
import numpy as np

class ExactCosineIndex:
    def __init__(self, embeddings: np.ndarray, ids: list[str]):
        """embeddings: (n, d) float32, already normalized."""
        self.embeddings = embeddings.astype(np.float32)
        self.ids = np.array(ids)
    
    def search(self, query: np.ndarray, k: int = 10) -> list[tuple[str, float]]:
        """query: (d,) or (1, d) normalized."""
        query = query.reshape(1, -1).astype(np.float32)
        scores = query @ self.embeddings.T  # (1, n)
        top_k_idx = np.argpartition(-scores[0], k)[:k]
        top_k_idx = top_k_idx[np.argsort(-scores[0, top_k_idx])]
        return [(self.ids[i], float(scores[0, i])) for i in top_k_idx]

Benchmarks on a modern CPU: ~50k vectors × 384 dims ≈ 2-3ms per query. Scales linearly. Fine for internal tools, not for user-facing latency budgets.

Path B: FAISS (100k – 10M vectors)

# faiss_index.py
import faiss
import numpy as np

class FaissCosineIndex:
    def __init__(self, embeddings: np.ndarray, ids: list[str]):
        """embeddings: (n, d) float32, already normalized."""
        self.d = embeddings.shape[1]
        self.ids = np.array(ids)
        
        # IndexFlatIP = exact inner product (cosine for normalized vecs)
        self.index = faiss.IndexFlatIP(self.d)
        self.index.add(embeddings.astype(np.float32))
    
    def search(self, query: np.ndarray, k: int = 10) -> list[tuple[str, float]]:
        query = query.reshape(1, -1).astype(np.float32)
        scores, idx = self.index.search(query, k)
        return [(self.ids[i], float(scores[0, j])) for j, i in enumerate(idx[0])]

FAISS IndexFlatIP is exact cosine search with SIMD acceleration. ~1M vectors × 768 dims ≈ 1-2ms. Add IndexIVFFlat or IndexHNSWFlat for approximate search at 10M+ scale.

Path C: Vector databases (10M+ vectors, filtered search, multi-tenant)

Use Pinecone, Weaviate, Qdrant, or Milvus. They all expose cosine similarity as a first-class metric. Example with Qdrant:

# qdrant_example.py
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
import numpy as np

client = QdrantClient(":memory:")  # or host:port
COLLECTION = "docs"

client.recreate_collection(
    collection_name=COLLECTION,
    vectors_config=VectorParams(size=384, distance=Distance.COSINE),
)

def upsert(embeddings: np.ndarray, ids: list[str], payloads: list[dict]):
    points = [
        PointStruct(id=id_, vector=vec.tolist(), payload=payload)
        for id_, vec, payload in zip(ids, embeddings, payloads)
    ]
    client.upsert(collection_name=COLLECTION, points=points)

def search(query: np.ndarray, k: int = 10, filter_: dict | None = None):
    hits = client.search(
        collection_name=COLLECTION,
        query_vector=query.tolist(),
        limit=k,
        query_filter=filter_,
    )
    return [(hit.id, hit.score, hit.payload) for hit in hits]

Critical: When using a vector database, do not normalize client-side if the database expects raw vectors. Qdrant with Distance.COSINE normalizes internally. Pinecone’s cosine metric does the same. Check your provider’s docs — double normalization breaks scores.

Step 7: Verify correctness end-to-end

Before shipping, run these sanity checks:

# verify.py
import numpy as np
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# 1. Self-similarity = 1.0
vec = model.encode(["test sentence"], normalize_embeddings=True)
assert np.isclose(cosine_similarity(vec, vec)[0, 0], 1.0, atol=1e-6)

# 2. Symmetry
a = model.encode(["first"], normalize_embeddings=True)
b = model.encode(["second"], normalize_embeddings=True)
sim_ab = cosine_similarity(a, b)[0, 0]
sim_ba = cosine_similarity(b, a)[0, 0]
assert np.isclose(sim_ab, sim_ba, atol=1e-6)

# 3. Known paraphrases > known unrelated
para = model.encode(["cat on mat", "kitten on rug"], normalize_embeddings=True)
unrel = model.encode(["cat on mat", "stock market"], normalize_embeddings=True)
assert cosine_similarity(para[0:1], para[1:2])[0, 0] > cosine_similarity(unrel[0:1], unrel[1:2])[0, 0]

# 4. Range check
scores = cosine_similarity(para, unrel)
assert scores.min() >= -1.0 - 1e-6 and scores.max() <= 1.0 + 1e-6

print("All verification checks passed.")

Run it. If any assertion fails, your embedding model, normalization, or similarity function has a bug — fix it before indexing.

Step 8: Monitor in production

Cosine similarity semantic closeness degrades silently when upstream embeddings shift. Track these metrics:

# metrics.py
import numpy as np
from dataclasses import dataclass

@dataclass
class SimilarityMetrics:
    mean_score: float
    p50_score: float
    p95_score: float
    zero_vector_rate: float
    norm_deviation: float  # mean | ||v|| - 1.0 |

def compute_metrics(embeddings: np.ndarray, query_embeddings: np.ndarray) -> SimilarityMetrics:
    """Call periodically on a sample of production traffic."""
    scores = cosine_similarity(query_embeddings, embeddings)
    norms = np.linalg.norm(embeddings, axis=1)
    return SimilarityMetrics(
        mean_score=float(scores.mean()),
        p50_score=float(np.percentile(scores, 50)),
        p95_score=float(np.percentile(scores, 95)),
        zero_vector_rate=float((norms < 1e-6).mean()),
        norm_deviation=float(np.abs(norms - 1.0).mean()),
    )

Alert on:

• norm_deviation > 0.01 → embedding pipeline producing non-normalized vectors
• zero_vector_rate > 0 → failed embedding calls reaching the index
• p50_score drifting by > 0.05 week-over-week → model or data distribution shift

Summary checklist

• Embeddings are L2 normalized (verify norms ≈ 1.0)
• Cosine similarity computed via dot product on normalized vectors
• Thresholds calibrated on representative labeled pairs
• Zero vectors handled explicitly (drop or log)
• Index choice matches scale: NumPy < 100k, FAISS 100k–10M, vector DB > 10M
• No double normalization (client + server)
• Verification tests pass in CI
• Production metrics tracked and alerted

Cosine similarity is deceptively simple — a dot product on normalized vectors. The complexity lives in the surrounding system: normalization guarantees, index selection, threshold calibration, and drift detection. Get those right and you have a semantic similarity backbone that scales.