Comparing vector stores for Semantic Kernel memory

When you need to compare vector stores semantic kernel memory backends, the IMemoryStore interface in Semantic Kernel makes them look interchangeable. They are not. The abstraction hides connection details but leaks through latency, cost structure, and query capabilities the moment you scale past a toy dataset.

How Semantic Kernel memory works

Semantic Kernel persists memories as MemoryRecord objects containing an embedding vector, a string key, arbitrary metadata, and a timestamp. The IMemoryStore implementation handles collection creation, upserts, and nearest-neighbor search. You register exactly one store with the MemoryBuilder, and the rest of your code calls memory.SaveInformationAsync or memory.SearchAsync without caring about the backend.

var memory = new MemoryBuilder()
    .WithOpenAIEmbeddings("text-embedding-3-small", Environment.GetEnvironmentVariable("OPENAI_KEY"))
    .WithMemoryStore(qdrantStore)
    .Build();

The embedding generator is a separate concern. If you want resilience against provider outages, an OpenAI-compatible gateway such as n4n.ai can front multiple embedding models and automatically fall back when a provider is rate-limited or degraded.

Collections in SK are flat namespaces. There is no built-in tenant isolation beyond prefixing collection names yourself. That matters when you compare vector stores semantic kernel memory options that differ in multi-tenant performance.

Dimensions that separate the stores

Before the table, name the axes that matter in production:

• Capabilities: hybrid search (vector + keyword), metadata filtering expressiveness, payload indexing, index rebuild cost.
• Cost model: open-source self-hosted (RAM/CPU) vs managed per-operation or per-node pricing.
• Latency/throughput: p99 query time under load, batch ingest speed, cold-start penalty.
• Ergonomics: local dev story, Docker image size, C# connector maturity, config surface.
• Ecosystem: surrounding tooling, admin UI, backup/restore story, cloud integrations.
• Limits: max vector dimensions, record size caps, scaling ceiling, quota enforcement.

The candidates

• Azure AI Search: Managed service, deep Azure integration, supports vector + keyword hybrid and semantic reranking. Requires an index definition upfront.
• Qdrant: Rust-based open-source vector DB with a managed cloud; strong payload filtering and quantization. HTTP API, no external deps.
• Redis: In-memory store via RediSearch module; brutal low latency if you already run Redis. Vectors stored as part of hash structures.
• Chroma: Lightweight embedded or client-server store, popular for notebooks and prototypes. Persists to disk with SQLite or runs as a container.
• PostgreSQL (pgvector): Extension turning your existing DB into a vector store. Supports IVFFlat and HNSW indexes.
• Pinecone: Fully managed serverless vector DB, zero infra, usage-based billing, namespaces for isolation.

Head-to-head

Wiring the connectors

Each store has a distinct connection pattern. Below are minimal C# snippets from the Semantic Kernel connectors. The dimension argument must match your embedding model or ingest fails.

// Qdrant
var qdrant = new QdrantMemoryStore("http://localhost:6333", 1536);

// Redis
var redis = new RedisMemoryStore("localhost:6379", 1536);

// Azure AI Search
var azure = new AzureAISearchMemoryStore(searchEndpoint, apiKey, "index");

// PostgreSQL
var pg = new PostgresMemoryStore(connString, 1536);

// Chroma
var chroma = new ChromaMemoryStore("http://localhost:8000");

// Pinecone
var pinecone = new PineconeMemoryStore(apiKey, "index", 1536);

The SK connectors are thin wrappers. They do not expose store-specific tuning like Qdrant’s quantization or pgvector’s ef_search. You will drop to the native client for that.

Metadata filtering and hybrid search gaps

Semantic Kernel’s MemoryQueryResult returns records but the IMemoryStore search API only supports a loose minRelevanceScore and collection scoping. If you need to filter by metadata (e.g., userId), only some backends can push that down efficiently.

Qdrant and Azure AI Search index payloads and support server-side filtering; the SK connectors pass along basic equality filters for those. Redis and Chroma do filtering in memory after vector search, which is fine for small collections but degrades past tens of thousands of records. PostgreSQL can join with relational tables, giving you the richest filter language.

When you compare vector stores semantic kernel memory for a multi-tenant app, verify the connector’s filter pass-through before committing.

Latency and throughput realities

Volatile and Redis win for raw speed because everything is in memory. Qdrant and Chroma are close behind if colocated. PostgreSQL with HNSW is surprisingly competitive for read-heavy workloads but ingesting millions of rows will lock your table without careful CREATE INDEX concurrency settings (max_parallel_maintenance_workers).

Azure AI Search and Pinecone add network hops. Pinecone serverless can stall on cold namespaces; Azure adds TLS and regional routing. For a user-facing chat with <100ms budget, run the store in the same VPC as your app. Benchmark with your own embedding size; 1536-d vectors behave differently than 384-d.

Cost and operational limits

Self-hosting Qdrant or Chroma is nearly free until RAM runs out. Redis is cheap if you already cache there; otherwise the memory premium stings. PostgreSQL piggybacks on infra you likely have, but vector columns bloat storage and backups.

Managed options trade money for zero ops. Pinecone bills per vector read/write and storage; Azure bills per search unit whether you use it or not. Both cap dimensions and total vectors per index—read the quota docs before committing. Qdrant Cloud bills per node but lets you enable disk-backed storage to cut RAM cost.

Index tuning you will eventually touch

• Qdrant: set hnsw_config and consider scalar/products quantization to shrink RAM.
• pgvector: choose HNSW over IVFFlat for dynamic data; tune m and ef_construction.
• Redis: use FT.CREATE with DISTANCE_METRIC and a numeric field for metadata.
• Azure: pick vectorSearch.profile and enable semantic configuration for rerank.

SK does not surface these; you configure them out-of-band.

Which to choose

Local prototype or test harness Use VolatileMemoryStore or Chroma. Zero external dependencies, fast iteration, no Docker needed for Chroma embedded.

Existing PostgreSQL shop Enable pgvector. You avoid a new system and get SQL joins for free. Accept the index maintenance overhead and plan migration scripts.

Already running Redis Use the Redis connector. Sub-millisecond lookups and you already monitor it. Watch memory headroom.

Azure-centric production Azure AI Search gives you hybrid search and compliance hooks with minimal YAML. Pay for the search unit and move on.

High-scale, no-ops team Qdrant Cloud or Pinecone. Qdrant if you want control and a Rust binary; Pinecone if you never want to touch a node.

Embedding resilience Regardless of store, generate embeddings through a gateway that supports fallback. That decouples memory from a single model provider outage.

If you compare vector stores semantic kernel memory for a greenfield project, default to the system you already operate. The abstraction will not save you from a mismatched cost or latency profile.