Testing Semantic Kernel plugins locally is the fastest way to catch hallucinated function signatures, serialization mismatches, and prompt-template drift before they hit production. This semantic kernel testing plugins locally tutorial walks through a complete local test strategy: unit-testing native functions in isolation, integration-testing the planner against a real kernel, and validating prompt templates without burning provider credits. You’ll leave with a test harness you can drop into CI.
Why local testing matters for plugins
Semantic Kernel plugins are just .NET classes or Python modules decorated with [KernelFunction] / @kernel_function. The framework discovers them via reflection, builds a function catalog, and hands that catalog to the planner. If a function’s signature doesn’t match what the planner expects — wrong parameter names, missing descriptions, incorrect return types — the planner either fails silently or generates invalid plans. Local tests catch these mismatches in milliseconds, not minutes.
The other reason to test locally: prompt templates. A plugin often ships with a skprompt.txt and config.json that define how the LLM should invoke the function. If the template references a parameter the function doesn’t accept, or if the function returns a shape the template doesn’t handle, you get runtime errors that are expensive to debug in a hosted environment.
Project structure for testability
Organize plugins so the kernel-hosting code is separate from the function implementations. A typical layout:
src/
Plugins/
Weather/
WeatherPlugin.cs # Native functions
Prompts/
GetForecast/
skprompt.txt
config.json
Kernel/
KernelBuilder.cs # DI registration, planner config
tests/
Unit/
WeatherPluginTests.cs
Integration/
PlannerIntegrationTests.csKeep the plugin class dependency-free. Inject ILogger, HttpClient, or configuration via constructor — not via kernel services. This lets you instantiate the plugin directly in unit tests without spinning up a Kernel instance.
Unit testing native functions
Native functions are plain methods. Test them like any other business logic: mock dependencies, assert return values, verify side effects.
// WeatherPlugin.cs
public sealed class WeatherPlugin
{
private readonly IWeatherApi _api;
private readonly ILogger<WeatherPlugin> _log;
public WeatherPlugin(IWeatherApi api, ILogger<WeatherPlugin> log)
{
_api = api;
_log = log;
}
[KernelFunction("get_forecast")]
[Description("Returns a 3-day forecast for a latitude/longitude.")]
public async Task<Forecast> GetForecastAsync(
[Description("Latitude in decimal degrees")] double lat,
[Description("Longitude in decimal degrees")] double lon,
CancellationToken ct = default)
{
_log.LogInformation("Fetching forecast for {Lat},{Lon}", lat, lon);
return await _api.GetForecastAsync(lat, lon, ct);
}
}// WeatherPluginTests.cs
public sealed class WeatherPluginTests
{
[Fact]
public async Task GetForecastAsync_ReturnsForecast_WhenApiSucceeds()
{
// Arrange
var mockApi = new Mock<IWeatherApi>();
mockApi.Setup(x => x.GetForecastAsync(47.6, -122.3, It.IsAny<CancellationToken>()))
.ReturnsAsync(new Forecast { Days = new[] { new DayForecast { High = 72, Low = 55 } } });
var logger = Mock.Of<ILogger<WeatherPlugin>>();
var plugin = new WeatherPlugin(mockApi.Object, logger);
// Act
var result = await plugin.GetForecastAsync(47.6, -122.3);
// Assert
Assert.NotNull(result);
Assert.Single(result.Days);
Assert.Equal(72, result.Days[0].High);
mockApi.Verify(x => x.GetForecastAsync(47.6, -122.3, It.IsAny<CancellationToken>()), Times.Once);
}
[Fact]
public async Task GetForecastAsync_Throws_WhenApiThrows()
{
var mockApi = new Mock<IWeatherApi>();
mockApi.Setup(x => x.GetForecastAsync(It.IsAny<double>(), It.IsAny<double>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new HttpRequestException("upstream down"));
var plugin = new WeatherPlugin(mockApi.Object, Mock.Of<ILogger<WeatherPlugin>>());
await Assert.ThrowsAsync<HttpRequestException>(() => plugin.GetForecastAsync(0, 0));
}
}No kernel, no planner, no LLM. Pure unit test. Run it in milliseconds on every build.
Testing prompt-template functions
Prompt-template functions live in skprompt.txt and are registered via kernel.ImportPluginFromPromptDirectory. The template renders with the kernel’s PromptTemplateEngine (Handlebars by default). Test the rendering in isolation, then test the full invoke path with a fake LLM.
Render test
// PromptRenderTests.cs
public sealed class PromptRenderTests
{
[Fact]
public void GetForecastPrompt_RendersCorrectly()
{
// Arrange
var template = File.ReadAllText("Plugins/Weather/Prompts/GetForecast/skprompt.txt");
var engine = new HandlebarsPromptTemplateEngine();
var args = new KernelArguments { ["location"] = "Seattle", ["days"] = 3 };
// Act
var rendered = engine.Render(template, args);
// Assert
Assert.Contains("Seattle", rendered);
Assert.Contains("3", rendered);
Assert.DoesNotContain("{{", rendered); // no unrendered placeholders
}
}Invoke test with a test double
Semantic Kernel ships Microsoft.SemanticKernel.TestUtils with FakeChatCompletionService. Use it to verify the function calls the LLM with the right prompt and parses the response correctly.
// PromptFunctionIntegrationTests.cs
public sealed class PromptFunctionIntegrationTests
{
[Fact]
public async Task GetForecastFunction_ReturnsParsedForecast()
{
// Arrange
var kernel = Kernel.CreateBuilder()
.AddFakeChatCompletion("{\"high\":72,\"low\":55,\"condition\":\"sunny\"}")
.Build();
var plugin = kernel.ImportPluginFromPromptDirectory(
"Plugins/Weather/Prompts/GetForecast",
"Weather");
// Act
var result = await kernel.InvokeAsync(
plugin["GetForecast"],
new KernelArguments { ["location"] = "Seattle", ["days"] = 1 });
// Assert
var forecast = result.GetValue<Forecast>();
Assert.Equal(72, forecast.High);
Assert.Equal(55, forecast.Low);
Assert.Equal("sunny", forecast.Condition);
}
}FakeChatCompletionService returns a fixed response. You control the JSON shape. This validates deserialization without hitting a real model.
Integration testing the planner
The planner (FunctionCallingStepwisePlanner, HandlebarsPlanner, or the newer AutoFunctionInvocation) orchestrates multiple functions. Test it against a real kernel with fake services so you verify the plan shape, not just individual functions.
// PlannerIntegrationTests.cs
public sealed class PlannerIntegrationTests
{
[Fact]
public async Task Planner_CallsWeatherThenEmail_ForUserRequest()
{
// Arrange
var kernel = Kernel.CreateBuilder()
.AddFakeChatCompletion(
// First call: planner emits function call
"{\"tool_calls\":[{\"id\":\"call_1\",\"function\":{\"name\":\"Weather-GetForecast\",\"arguments\":\"{\\\"lat\\\":47.6,\\\"lon\\\":-122.3}\"}}]}",
// Second call: planner emits final answer after function returns
"The forecast for Seattle is sunny, 72°F."
)
.Build();
// Register real plugins (they'll use fake HTTP clients you provide)
kernel.Plugins.AddFromType<WeatherPlugin>();
kernel.Plugins.AddFromType<EmailPlugin>();
var planner = new FunctionCallingStepwisePlanner();
// Act
var result = await planner.ExecuteAsync(kernel, "Email me the Seattle forecast");
// Assert
Assert.Contains("sunny", result.FinalAnswer);
Assert.Contains("72", result.FinalAnswer);
}
}The fake chat completion returns a sequence of responses. The first response simulates the planner deciding to call Weather-GetForecast. The kernel executes that function (using your real plugin code with mocked dependencies), then feeds the result back to the planner, which produces the second response. This exercises the full loop: plan → execute → observe → replan → answer.
Common pitfalls
Parameter name mismatch
The planner matches function parameters by name. If your C# parameter is latitude but the JSON schema says lat, the planner sends lat and the binder drops it. Unit tests that invoke the function directly won’t catch this — only planner integration tests do.
Fix: make the [Description] attribute match the JSON property name, or use [JsonPropertyName("lat")] on the parameter.
[KernelFunction("get_forecast")]
public async Task<Forecast> GetForecastAsync(
[Description("Latitude in decimal degrees")]
[JsonPropertyName("lat")] double latitude,
...
)Missing function descriptions
The planner relies on Description attributes to decide which function to call. If you omit them, the planner guesses — often wrong. Enforce descriptions via a Roslyn analyzer or a unit test that reflects over all plugin methods:
[Fact]
public void AllPluginFunctions_HaveDescriptions()
{
var pluginTypes = Assembly.GetExecutingAssembly()
.GetTypes()
.Where(t => t.GetCustomAttribute<KernelFunctionAttribute>() != null);
foreach (var type in pluginTypes)
{
var methods = type.GetMethods(BindingFlags.Public | BindingFlags.Instance)
.Where(m => m.GetCustomAttribute<KernelFunctionAttribute>() != null);
foreach (var method in methods)
{
var desc = method.GetCustomAttribute<DescriptionAttribute>();
Assert.NotNull(desc);
Assert.False(string.IsNullOrWhiteSpace(desc.Description),
$"{type.Name}.{method.Name} missing description");
}
}
}Serialization surprises
KernelArguments uses System.Text.Json with default options (camelCase, no enums as strings). If your function returns a type with JsonConverter attributes or non-standard naming, the planner may serialize it differently than your unit test expects. Test the actual serialization path:
[Fact]
public void Forecast_SerializesAsExpected()
{
var forecast = new Forecast { High = 72, Low = 55, Condition = "sunny" };
var json = JsonSerializer.Serialize(forecast, KernelJsonSerializerContext.Default.Forecast);
Assert.Equal("{\"high\":72,\"low\":55,\"condition\":\"sunny\"}", json);
}Prompt template drift
Prompt templates live in .txt files. They’re easy to edit without updating the corresponding function signature. Add a test that validates every skprompt.txt references only parameters that exist on the target function:
[Fact]
public void PromptTemplates_ReferenceValidParameters()
{
var promptDirs = Directory.GetDirectories("Plugins", "Prompts", SearchOption.AllDirectories);
foreach (var dir in promptDirs)
{
var prompt = File.ReadAllText(Path.Combine(dir, "skprompt.txt"));
var config = JsonSerializer.Deserialize<PromptConfig>(File.ReadAllText(Path.Combine(dir, "config.json")));
var pluginType = Assembly.GetExecutingAssembly()
.GetTypes()
.First(t => t.Name == config.PluginName);
var method = pluginType.GetMethod(config.FunctionName);
var paramNames = method.GetParameters().Select(p => p.Name).ToHashSet();
// Extract {{variable}} references from prompt
var referenced = Regex.Matches(prompt, @"\{\{(\w+)\}\}")
.Select(m => m.Groups[1].Value)
.ToHashSet();
foreach (var refName in referenced)
{
Assert.Contains(refName, paramNames);
}
}
}Running in CI
Add a test project to your pipeline. No secrets required — everything runs against fakes.
# .github/workflows/test.yml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
- run: dotnet restore
- run: dotnet build --no-restore
- run: dotnet test --no-build --logger "trx;LogFileName=results.trx"
- uses: actions/upload-artifact@v4
if: always()
with:
name: test-results
path: **/*.trxRun on every PR. The feedback loop stays under 30 seconds.
Debugging locally with a real model (when you must)
Sometimes you need to see what a real model produces. Use a local model via Ollama or LM Studio pointed at your kernel. Configure the kernel to use the local endpoint:
var kernel = Kernel.CreateBuilder()
.AddOpenAIChatCompletion(
modelId: "llama3.1:8b",
apiKey: "ollama", // dummy
endpoint: new Uri("http://localhost:11434/v1"))
.Build();This lets you iterate on prompt templates with real completions while keeping the rest of the test harness local. When the template stabilizes, swap back to FakeChatCompletionService for CI.
If you’re routing through a gateway that normalizes provider APIs — n4n.ai exposes a single OpenAI-compatible endpoint across 240+ models with automatic fallback — you can point the local kernel at that gateway during manual testing and swap the base URL without changing client code.
Summary checklist
- Unit test every native function with mocked dependencies
- Render-test every prompt template with
HandlebarsPromptTemplateEngine - Invoke-test prompt functions with
FakeChatCompletionService - Integration-test the planner with a multi-response fake sequence
- Enforce
Descriptionattributes on all[KernelFunction]methods - Validate prompt-template parameter references against function signatures
- Verify JSON serialization matches planner expectations
- Run the full suite in CI on every PR
- Use a local model (Ollama/LM Studio) only for prompt authoring, not CI
This semantic kernel testing plugins locally tutorial gives you a complete local test strategy. The key insight: treat the kernel as a composition root you can swap. Test functions in isolation, test the planner with fakes, and only reach for a real model when you’re tuning prompts. Your CI stays fast, your plugins stay reliable, and you stop burning credits on broken plans.