What are embeddings in AI, explained simply?

Embeddings are dense vector representations that map discrete inputs — words, sentences, images, or any structured data — into a continuous high-dimensional space where semantic similarity corresponds to geometric proximity. When you ask what are embeddings in ai, the answer is fundamentally about translation: converting symbolic data into numerical coordinates that preserve relationships. This translation is what lets neural networks reason over language, code, and multimodal inputs using the same mathematical machinery.

How embeddings work

At the mechanical level, an embedding model is a function $f: \mathcal{X} \to \mathbb{R}^d$ that takes an input $x$ from a discrete vocabulary or structured domain and outputs a $d$-dimensional vector. The dimensionality $d$ typically ranges from 256 to 4096 for modern models, though smaller dimensions work for constrained vocabularies.

The training objective shapes what the geometry means. For word-level embeddings like Word2Vec or GloVe, the objective is co-occurrence prediction: words appearing in similar contexts get pulled closer together. For sentence transformers (SBERT, E5, BGE), the objective is contrastive learning on pairs — minimizing distance for semantically equivalent sentences and maximizing it for dissimilar ones. For multimodal models like CLIP, the objective aligns image and text embeddings in a shared space.

# Conceptual interface — any embedding model implements this
class EmbeddingModel:
    def encode(self, texts: list[str]) -> np.ndarray:
        """Return shape (n, d) float32 array."""
        ...

    def similarity(self, a: np.ndarray, b: np.ndarray) -> float:
        """Cosine similarity between two vectors."""
        return (a @ b) / (np.linalg.norm(a) * np.linalg.norm(b))

The critical property: distance in vector space approximates semantic distance. Cosine similarity is the standard metric because magnitude often correlates with input length or frequency rather than meaning. Normalize your vectors to unit length and dot product becomes cosine similarity.

# Normalize once after encoding, then use fast dot products
embeddings = model.encode(texts)
embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
similarities = embeddings @ query_embedding.T  # (n,) array of cosine scores

Why embeddings matter for engineers

Embeddings are the connective tissue between symbolic data and neural computation. They enable:

Semantic search and retrieval. Instead of keyword matching (BM25, inverted indices), you encode queries and documents into the same space and retrieve by vector similarity. This handles synonyms, paraphrases, and conceptual matches that lexical search misses.

Retrieval-augmented generation (RAG). The standard RAG pipeline: chunk documents → embed chunks → store in vector database → at query time, embed the question → retrieve top-k chunks → feed to LLM. The embedding quality directly bounds retrieval recall, which bounds answer quality.

Clustering and classification. Embeddings turn unstructured data into feature vectors for downstream ML. Cluster customer support tickets to discover emerging issues. Classify code snippets by language or framework. Detect anomalies in log embeddings.

Recommendation and personalization. User and item embeddings (from collaborative filtering or content-based models) enable nearest-neighbor lookup at serving time. Two-tower architectures learn separate user/item encoders trained on interaction data.

Cross-lingual and cross-modal transfer. Models like LaBSE or CLIP align multiple languages or modalities in one space. A single embedding can serve search across 100+ languages or retrieve images from text queries.

Concrete example: building a semantic search index

Here’s a minimal working pipeline using sentence-transformers and faiss — the same pattern that scales to millions of vectors with approximate nearest neighbor (ANN) indices.

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

# 1. Load a strong general-purpose encoder (384-dim, fast, multilingual)
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# 2. Your corpus — in practice, chunk long documents first
documents = [
    "How to configure PostgreSQL connection pooling with PgBouncer",
    "Optimizing Redis memory usage for session stores",
    "Kubernetes pod disruption budgets explained",
    "Debugging memory leaks in Go services",
    "Setting up distributed tracing with OpenTelemetry",
]

# 3. Encode and normalize
embeddings = model.encode(documents, convert_to_numpy=True, normalize_embeddings=True)
# shape: (5, 384), float32, unit length

# 4. Build a flat IP index (exact search, fine for <100k vectors)
index = faiss.IndexFlatIP(embeddings.shape[1])
index.add(embeddings.astype(np.float32))

# 5. Query
query = "reduce memory footprint of redis"
query_vec = model.encode([query], normalize_embeddings=True).astype(np.float32)
scores, indices = index.search(query_vec, k=3)

for score, idx in zip(scores[0], indices[0]):
    print(f"{score:.3f}  {documents[idx]}")

Output:

0.742  Optimizing Redis memory usage for session stores
0.511  Debugging memory leaks in Go services
0.489  How to configure PostgreSQL connection pooling with PgBouncer

The top result matches intent (“reduce memory footprint”) despite zero keyword overlap with “Optimizing Redis memory usage.” That’s the embedding doing its job.

Chunking matters more than you think

Embedding a 50-page PDF as one vector dilutes signal. Standard practice: split into overlapping chunks (500-1000 tokens, 10-20% overlap), embed each chunk, store chunk metadata (source, page, section) alongside the vector. At retrieval time, you get precise citations.

def chunk_text(text: str, chunk_size: int = 512, overlap: int = 50) -> list[str]:
    """Naive character-based chunking. Use a tokenizer for production."""
    chunks = []
    start = 0
    while start < len(text):
        end = min(start + chunk_size, len(text))
        chunks.append(text[start:end])
        start += chunk_size - overlap
    return chunks

Production systems use token-aware chunkers (by sentence, by markdown header, by code block) and often embed multiple granularities — sentence-level for precision, section-level for context.

Common misconceptions

“Embeddings are just compressed one-hot vectors”

One-hot vectors are sparse, orthogonal, and carry zero semantic information — every word is equidistant from every other. Embeddings are dense, learned, and structured: the geometry encodes relationships. King - Man + Woman ≈ Queen works because the training objective forces relational structure, not because of compression.

“Higher dimension always means better quality”

Diminishing returns hit hard past 1024 dimensions for general-purpose text. Larger models (768d, 1024d, 4096d) capture finer distinctions but cost more to store, index, and search. For many retrieval tasks, a well-trained 384-dim model (MiniLM, BGE-small) outperforms a poorly trained 1024-dim model. Measure recall@k on your data, not dimension count.

“Cosine similarity is the only metric”

Cosine is standard for normalized vectors. But if you don’t normalize, dot product magnitude carries information — longer or more specific texts often have larger norms. Some retrieval systems use dot product intentionally. Euclidean distance works on normalized vectors (monotonic with cosine) but is slower to compute. For ANN indices, the index type constrains the metric: HNSW supports cosine/IP/L2; IVF typically uses L2.

“One embedding model works for everything”

Domain mismatch kills retrieval. A model trained on web text (MS MARCO, Wikipedia) underperforms on legal contracts, medical records, or code. Fine-tune or select domain-adapted models: bge-code-v1 for code, legal-bert embeddings for contracts, pubmedbert for biomedical text. Evaluate on your queries before committing.

“Embeddings are deterministic”

Most embedding models are deterministic at inference (no dropout, fixed weights). But floating-point non-determinism across hardware, library versions, or batch sizes can produce tiny differences. If you need bitwise reproducibility for caching or testing, pin PyTorch/ONNX versions, set torch.use_deterministic_algorithms(True), and avoid dynamic batching.

“Vector databases are magic”

Vector databases (Pinecone, Weaviate, Qdrant, Milvus, Chroma) are ANN indices with persistence, filtering, and horizontal scaling. They don’t improve embedding quality. If your recall is low, the fix is better embeddings, better chunking, hybrid search (BM25 + vector), or query expansion — not a different index. The index only affects latency/recall tradeoffs at scale.

Choosing an embedding model in 2024

The landscape settles into a few tiers:

Instruction-tuned models accept a task prefix: "query: find python async patterns" vs "passage: async def fetch_data():". This improves separation between query and document distributions. Use the matching prefix at query and index time.

# E5-style instruction tuning
query_vec = model.encode(["query: " + user_question], normalize_embeddings=True)
doc_vecs = model.encode(["passage: " + d for d in documents], normalize_embeddings=True)

Evaluation: measure what matters

Don’t trust leaderboard numbers. Build a small labeled set (50-200 queries with relevant doc IDs) and measure:

• Recall@k: fraction of queries where at least one relevant doc appears in top-k
• nDCG@k: ranking quality, penalizes relevant docs buried lower
• Latency@p99: end-to-end p99 including encoding, search, reranking

def recall_at_k(retrieved: list[list[int]], relevant: list[set[int]], k: int) -> float:
    hits = sum(1 for r, rel in zip(retrieved, relevant) if any(doc in rel for doc in r[:k]))
    return hits / len(relevant)

Reranking with a cross-encoder (e.g., cross-encoder/ms-marco-MiniLM-L-6-v2) typically adds 5-15 points of nDCG@10 over bi-encoder retrieval alone, at the cost of scoring each candidate pair. Standard pattern: retrieve 50-100 with bi-encoder, rerank top 10-20 with cross-encoder, return top 5-10 to the LLM.

Storage and serving considerations

Quantization. Float32 (4 bytes/dim) → int8 (1 byte/dim) via scalar quantization cuts memory 4x with <1% recall loss for most models. Product quantization (PQ) compresses further for billion-scale indices. Faiss supports both; most vector databases handle it transparently.

# Scalar quantization in faiss
index = faiss.IndexFlatIP(d)
index = faiss.index_factory(d, "IVF1024,PQ32x8")  # IVF + 32 subquantizers, 8 bits each
index.train(embeddings)
index.add(embeddings)

Metadata filtering. Real queries need filters: “show me Redis docs from the last 6 months” or “only Python files.” Store filterable fields (timestamp, language, tenant_id) alongside vectors. Most vector databases support pre-filter (filter then search) and post-filter (search then filter); pre-filter is more accurate, post-filter is faster. Hybrid: pre-filter on high-cardinality fields, post-filter on low-cardinality.

Updates and deletion. Append-only indices are simple; deleting vectors from HNSW or IVF requires rebuild or tombstone compaction. For mutable corpora, consider a write-ahead log + periodic reindex, or a database with native upsert (Qdrant, Weaviate).

When not to use embeddings

• Exact matching required: IDs, SKUs, error codes, regex patterns — use inverted index or hash lookup.
• Structured queries: “price < 100 AND category = ‘electronics’” — use SQL/OLAP.
• Tiny corpus (<1k docs): BM25 or even linear scan with cross-encoder is simpler and often better.
• Explainability mandated: If you must show why a doc matched, lexical highlights beat “vector similarity said so.”

Closing thought

Embeddings are not magic — they’re a learned projection that makes semantic structure geometrically accessible. The engineering leverage comes from treating them as a first-class data type: version your embedding model like you version code, evaluate on your distribution, monitor drift in production, and iterate. The vector is just the interface; the quality of the retrieval system is the product of chunking strategy, embedding model, index configuration, reranking, and evaluation discipline.