This semantic kernel agent memory tutorial walks you through adding persistent, queryable memory to a Semantic Kernel agent. You will build a working agent that recalls facts across sessions, using a local vector store for embeddings and the built-in memory skill for retrieval. The code targets Semantic Kernel 1.19+ with .NET 8.
Prerequisites
- .NET 8 SDK installed
- An OpenAI-compatible API key (OpenAI, Azure OpenAI, or a gateway like n4n.ai)
- Basic familiarity with Semantic Kernel concepts: kernels, plugins, and planners
Create a new console project and add the required packages:
dotnet new console -n SKAgentMemory
cd SKAgentMemory
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Plugins.Memory
dotnet add package Microsoft.SemanticKernel.Connectors.Memory.Qdrant
dotnet add package Microsoft.Extensions.VectorData.QdrantQdrant runs locally in Docker for this tutorial. Start it now:
docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrantProject structure
SKAgentMemory/
├── Program.cs
├── Models/
│ └── MemoryRecord.cs
├── Services/
│ └── MemoryService.cs
└── Agents/
└── MemoryAgent.csStep 1: Define the memory record
Semantic Kernel’s vector store abstraction expects a record class with vector and data properties. Create Models/MemoryRecord.cs:
namespace SKAgentMemory.Models;
public sealed class MemoryRecord
{
[VectorStoreRecordKey]
public string Id { get; set; } = Guid.NewGuid().ToString();
[VectorStoreRecordVector(1536)]
public ReadOnlyMemory<float> Embedding { get; set; }
[VectorStoreRecordData]
public string Content { get; set; } = string.Empty;
[VectorStoreRecordData]
public string Collection { get; set; } = "default";
[VectorStoreRecordData]
public DateTimeOffset Timestamp { get; set; } = DateTimeOffset.UtcNow;
[VectorStoreRecordData]
public Dictionary<string, string> Metadata { get; set; } = new();
}The VectorStoreRecordVector attribute specifies the embedding dimension. OpenAI’s text-embedding-3-small produces 1536 dimensions. Adjust if you use a different model.
Step 2: Build the memory service
The memory service wraps collection management, ingestion, and search. Create Services/MemoryService.cs:
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel.Memory;
using SKAgentMemory.Models;
namespace SKAgentMemory.Services;
public sealed class MemoryService : IDisposable
{
private readonly IVectorStore _vectorStore;
private readonly VectorStoreRecordCollection<string, MemoryRecord> _collection;
private readonly ITextEmbeddingGenerationService _embeddingService;
private bool _disposed;
public MemoryService(
IVectorStore vectorStore,
ITextEmbeddingGenerationService embeddingService,
string collectionName = "agent-memory")
{
_vectorStore = vectorStore;
_embeddingService = embeddingService;
_collection = _vectorStore.GetCollection<string, MemoryRecord>(collectionName);
}
public async Task InitializeAsync(CancellationToken ct = default)
{
await _collection.CreateCollectionIfNotExistsAsync(ct);
}
public async Task<string> StoreAsync(
string content,
string collection = "default",
Dictionary<string, string>? metadata = null,
CancellationToken ct = default)
{
var embedding = await _embeddingService.GenerateEmbeddingAsync(content, ct);
var record = new MemoryRecord
{
Content = content,
Collection = collection,
Embedding = embedding,
Metadata = metadata ?? new Dictionary<string, string>()
};
await _collection.UpsertAsync(record, ct);
return record.Id;
}
public async Task<IReadOnlyList<MemoryRecord>> SearchAsync(
string query,
int limit = 5,
double minRelevanceScore = 0.7,
string? collection = null,
CancellationToken ct = default)
{
var queryEmbedding = await _embeddingService.GenerateEmbeddingAsync(query, ct);
var filter = collection is not null
? new VectorSearchFilter().EqualTo(r => r.Collection, collection)
: null;
var results = await _collection.VectorizedSearchAsync(
queryEmbedding,
limit,
filter,
ct: ct);
var records = new List<MemoryRecord>();
await foreach (var result in results.Results)
{
if (result.Score >= minRelevanceScore)
{
records.Add(result.Record);
}
}
return records;
}
public void Dispose()
{
if (!_disposed)
{
_disposed = true;
}
}
}Step 3: Wire up the kernel and memory
Update Program.cs to configure the kernel, embedding service, Qdrant vector store, and the memory service:
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.VectorData;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.Qdrant;
using Microsoft.SemanticKernel.Embeddings;
using Microsoft.SemanticKernel.Memory;
using SKAgentMemory.Agents;
using SKAgentMemory.Services;
var builder = WebApplication.CreateBuilder(args);
// Replace with your endpoint and key
const string endpoint = "https://api.openai.com/v1";
const string apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY") ?? "your-key";
const string embeddingModel = "text-embedding-3-small";
const string chatModel = "gpt-4o-mini";
builder.Services.AddSingleton(sp =>
{
var qdrantClient = new QdrantClient("localhost", 6333);
return new QdrantVectorStore(qdrantClient);
});
builder.Services.AddSingleton<ITextEmbeddingGenerationService>(sp =>
new OpenAITextEmbeddingGenerationService(embeddingModel, apiKey, endpoint: endpoint));
builder.Services.AddSingleton<MemoryService>();
builder.Services.AddKernel()
.AddOpenAIChatCompletion(chatModel, apiKey, endpoint: endpoint);
builder.Services.AddSingleton<MemoryAgent>();
var app = builder.Build();
using var scope = app.Services.CreateScope();
var memoryService = scope.ServiceProvider.GetRequiredService<MemoryService>();
await memoryService.InitializeAsync();
var agent = scope.ServiceProvider.GetRequiredService<MemoryAgent>();
await agent.RunInteractiveLoopAsync();
app.Run();Install the Qdrant client package if missing:
dotnet add package Qdrant.ClientStep 4: Create the memory-enabled agent
The agent uses a KernelFunction that wraps the memory service for both storage and retrieval. Create Agents/MemoryAgent.cs:
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using SKAgentMemory.Services;
namespace SKAgentMemory.Agents;
public sealed class MemoryAgent
{
private readonly Kernel _kernel;
private readonly IChatCompletionService _chat;
private readonly MemoryService _memory;
private readonly ChatHistory _history = [];
public MemoryAgent(Kernel kernel, MemoryService memory)
{
_kernel = kernel;
_memory = memory;
_chat = kernel.GetRequiredService<IChatCompletionService>();
_history.AddSystemMessage("""
You are a helpful assistant with persistent memory.
When the user shares a fact, preference, or instruction, call the remember function.
When the user asks something that might be answered from memory, call the recall function first.
Always use the tools when appropriate. Do not hallucinate memory.
""");
RegisterMemoryFunctions();
}
private void RegisterMemoryFunctions()
{
var remember = KernelFunctionFactory.CreateFromMethod(
RememberAsync,
functionName: "remember",
);
var recall = KernelFunctionFactory.CreateFromMethod(
RecallAsync,
functionName: "recall",
);
_kernel.Plugins.AddFromFunctions("Memory", [remember, recall]);
}
public async Task<string> RememberAsync(
Kernel kernel,
[Description("The information to store")] string content,
[Description("Optional category: fact, preference, instruction, context")] string category = "fact",
[Description("Optional tags as comma-separated values")] string tags = "",
CancellationToken ct = default)
{
var metadata = new Dictionary<string, string>
{
["category"] = category,
["tags"] = tags
};
var id = await _memory.StoreAsync(content, "agent-memory", metadata, ct);
return $"Stored in memory (id: {id[..8]}). Category: {category}";
}
public async Task<string> RecallAsync(
Kernel kernel,
[Description("The query to search memory for")] string query,
[Description("Maximum results to return")] int limit = 5,
CancellationToken ct = default)
{
var results = await _memory.SearchAsync(query, limit, 0.65, "agent-memory", ct);
if (results.Count == 0)
{
return "No relevant memories found.";
}
var lines = results.Select((r, i) =>
$"{i + 1}. [{r.Metadata.GetValueOrDefault("category", "fact")}] {r.Content} (score: {r.Metadata.GetValueOrDefault("score", "N/A")})");
return "Relevant memories:\n" + string.Join("\n", lines);
}
public async Task RunInteractiveLoopAsync()
{
Console.WriteLine("Memory agent ready. Type 'exit' to quit.\n");
while (true)
{
Console.Write("> ");
var input = Console.ReadLine();
if (string.IsNullOrWhiteSpace(input) || input.Trim().Equals("exit", StringComparison.OrdinalIgnoreCase))
{
break;
}
_history.AddUserMessage(input);
var executionSettings = new OpenAIPromptExecutionSettings
{
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};
var result = await _chat.GetChatMessageContentAsync(_history, executionSettings, _kernel);
_history.Add(result);
Console.WriteLine($"\n{result}\n");
}
}
}Step 5: Run and verify
Set your API key and run:
export OPENAI_API_KEY=sk-...
dotnet runCheckpoint 1: Store a fact
> My name is Alex and I prefer TypeScript over Python for new projects.Expected output (abbreviated):
Stored in memory (id: a1b2c3d4). Category: fact
Got it. I've saved that you're Alex and prefer TypeScript for new projects.Checkpoint 2: Recall in a new turn
> What language should I use for the new API service?Expected output:
Relevant memories:
1. [fact] My name is Alex and I prefer TypeScript over Python for new projects.
Based on your preference, use TypeScript for the new API service.Checkpoint 3: Cross-session persistence
Stop the process (Ctrl+C), restart dotnet run, then ask:
> What's my name and language preference?Expected output:
Relevant memories:
1. [fact] My name is Alex and I prefer TypeScript over Python for new projects.
Your name is Alex and you prefer TypeScript over Python for new projects.The memory survives process restarts because Qdrant persists to disk by default.
Step 6: Add structured memory with collections
For multi-tenant or multi-domain agents, partition memory by collection. Update MemoryAgent.cs to accept a collection parameter:
public async Task<string> RememberAsync(
Kernel kernel,
[Description("The information to store")] string content,
[Description("Collection name for namespacing")] string collection = "default",
[Description("Optional category: fact, preference, instruction, context")] string category = "fact",
[Description("Optional tags as comma-separated values")] string tags = "",
CancellationToken ct = default)
{
var metadata = new Dictionary<string, string>
{
["category"] = category,
["tags"] = tags
};
var id = await _memory.StoreAsync(content, collection, metadata, ct);
return $"Stored in memory (id: {id[..8]}, collection: {collection}). Category: {category}";
}
public async Task<string> RecallAsync(
Kernel kernel,
[Description("The query to search memory for")] string query,
[Description("Collection to search")] string collection = "default",
[Description("Maximum results to return")] int limit = 5,
CancellationToken ct = default)
{
var results = await _memory.SearchAsync(query, limit, 0.65, collection, ct);
// ... same formatting as before
}Now the agent can keep separate memory namespaces per user, project, or domain without collisions.
Production considerations
Embedding model consistency
The vector store dimension must match the embedding model. If you switch from text-embedding-3-small (1536) to text-embedding-3-large (3072), you must recreate the collection or migrate vectors. Pin the model in configuration and treat dimension as a contract.
Relevance threshold tuning
The minRelevanceScore parameter (0.65 in the example) controls recall precision. Lower values return more results but increase noise. Log the scores during development to pick a threshold that matches your domain:
// During development, log scores to tune threshold
foreach (var result in results.Results)
{
_logger.LogDebug("Memory hit: score={Score}, content={Content}", result.Score, result.Record.Content[..50]);
}Token budget management
Each recalled memory consumes tokens in the prompt. Implement a token budget in the recall function:
public async Task<string> RecallAsync(/* ... */)
{
var results = await _memory.SearchAsync(query, limit, 0.65, collection, ct);
var budget = 1500; // tokens
var used = 0;
var lines = new List<string>();
foreach (var r in results)
{
var tokens = EstimateTokens(r.Content);
if (used + tokens > budget) break;
used += tokens;
lines.Add($"[{r.Metadata.GetValueOrDefault("category")}] {r.Content}");
}
return lines.Count > 0
? "Relevant memories:\n" + string.Join("\n", lines)
: "No relevant memories found within token budget.";
}Concurrency and scaling
Qdrant handles concurrent reads well. For write-heavy workloads, consider batching UpsertAsync calls or using a background queue. The MemoryService is thread-safe for concurrent reads; wraps writes in a semaphore if you observe contention.
Observability
Add structured logging around every memory operation:
_logger.LogInformation("Memory store: collection={Collection}, category={Category}, chars={Length}",
collection, category, content.Length);
_logger.LogInformation("Memory recall: query={Query}, results={Count}, latencyMs={Latency}",
query, results.Count, stopwatch.ElapsedMilliseconds);Correlate with request IDs to trace memory hits through the planner.
Next steps
- Replace the interactive loop with a proper API endpoint (ASP.NET Core Minimal API or Controller)
- Add a
forgetfunction for GDPR/right-to-be-forgotten compliance - Implement memory summarization: periodically condense old memories into higher-level concepts
- Explore Semantic Kernel’s
TextMemoryPluginfor a zero-code alternative if your needs are simpler
The pattern here — explicit remember and recall functions backed by a vector store — gives you full control over what enters memory, how it’s namespaced, and how it’s retrieved. That control matters when you move from demos to systems that users trust.