Semantic Kernel .NET tutorial: background services

This semantic kernel .net background service tutorial walks you through building a long-running hosted service that uses Semantic Kernel to process work asynchronously. You will create a .NET 8 worker project, register the kernel with dependency injection, add AI-powered plugins, and implement resilient message processing with proper cancellation and logging.

Prerequisites

• .NET 8 SDK installed
• An OpenAI-compatible API endpoint and key (or Azure OpenAI deployment)
• Basic familiarity with IHostedService and BackgroundService patterns
• A code editor (VS Code, Rider, or Visual Studio)

Verify your environment:

dotnet --version
# Should output 8.0.x

Create the worker project

Start with a minimal Worker Service template. This gives you the generic host, logging, and DI container pre-wired.

dotnet new worker -n SkBackgroundWorker
cd SkBackgroundWorker

Add the Semantic Kernel packages. We need the core library, the OpenAI connector, and the dependency injection extensions.

dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.OpenAI
dotnet add package Microsoft.Extensions.Hosting

Configure the kernel in Program.cs

Open Program.cs and replace the contents with a setup that registers a configured Kernel instance. We read the endpoint and key from configuration so they stay out of source control.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.SemanticKernel;
using SkBackgroundWorker.Services;

var builder = Host.CreateApplicationBuilder(args);

// Register Semantic Kernel with OpenAI chat completion
builder.Services.AddKernel()
    .AddOpenAIChatCompletion(
        modelId: builder.Configuration["OpenAI:Model"] ?? "gpt-4o-mini",
        apiKey: builder.Configuration["OpenAI:ApiKey"] ?? throw new InvalidOperationException("OpenAI:ApiKey not configured"),
        endpoint: new Uri(builder.Configuration["OpenAI:Endpoint"] ?? "https://api.openai.com/v1"),
        serviceId: "default"
    );

// Register our background service
builder.Services.AddHostedService<KernelWorker>();

var host = builder.Build();
host.Run()

Create appsettings.json (and appsettings.Development.json for local secrets) with your credentials:

{
  "OpenAI": {
    "Endpoint": "https://api.openai.com/v1",
    "Model": "gpt-4o-mini",
    "ApiKey": "sk-YOUR_KEY_HERE"
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.SemanticKernel": "Debug"
    }
  }
}

Checkpoint: Run dotnet run — the app should start, log “Worker running at: …”, and wait. Press Ctrl+C to stop.

Define a plugin for the background work

Background services typically process queue messages, scheduled tasks, or webhook payloads. We’ll simulate this with a plugin that classifies incoming text and extracts action items. Create Plugins/ClassificationPlugin.cs:

using Microsoft.SemanticKernel;

namespace SkBackgroundWorker.Plugins;

public sealed class ClassificationPlugin
{
    [KernelFunction("classify_intent")]
    [Description("Classify the intent of a user message into a predefined category.")]
    public string ClassifyIntent(
        [Description("The raw message text to classify")] string message,
        [Description("Comma-separated list of valid categories")] string categories)
    {
        // In a real scenario, you might call a lightweight model or rule engine here.
        // For this tutorial we return a deterministic stub so the pipeline is testable.
        var lowered = message.ToLowerInvariant();
        if (lowered.Contains("order") || lowered.Contains("buy")) return "sales";
        if (lowered.Contains("support") || lowered.Contains("help")) return "support";
        if (lowered.Contains("billing") || lowered.Contains("invoice")) return "billing";
        return "general";
    }

    [KernelFunction("extract_actions")]
    [Description("Extract actionable items from a message as a JSON array.")]
    public string ExtractActions(
        [Description("The message text")] string message)
    {
        // Stub implementation — replace with a prompt function for production.
        var actions = new List<string>();
        if (message.Contains("call", StringComparison.OrdinalIgnoreCase)) actions.Add("schedule_call");
        if (message.Contains("email", StringComparison.OrdinalIgnoreCase)) actions.Add("send_email");
        if (message.Contains("refund", StringComparison.OrdinalIgnoreCase)) actions.Add("process_refund");
        return System.Text.Json.JsonSerializer.Serialize(actions);
    }
}

Register the plugin in Program.cs (add after AddOpenAIChatCompletion):

builder.Services.AddSingleton<ClassificationPlugin>();
builder.Services.AddKernelPlugin<ClassificationPlugin>();

Implement the background service

Create Services/KernelWorker.cs. This class inherits from BackgroundService and overrides ExecuteAsync. We inject IKernel and ILogger, pull work from a simulated queue, invoke the kernel, and respect the cancellation token.

using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using SkBackgroundWorker.Plugins;

namespace SkBackgroundWorker.Services;

public sealed class KernelWorker : BackgroundService
{
    private readonly IKernel _kernel;
    private readonly ILogger<KernelWorker> _logger;
    private readonly ClassificationPlugin _plugin;
    private readonly TimeSpan _pollInterval = TimeSpan.FromSeconds(5);

    // Simulated in-memory queue for demo purposes
    private readonly Channel<string> _workQueue = Channel.CreateUnbounded<string>();

    public KernelWorker(IKernel kernel, ILogger<KernelWorker> logger, ClassificationPlugin plugin)
    {
        _kernel = kernel;
        _logger = logger;
        _plugin = plugin;
    }

    // Public method so you can enqueue work from controllers, other services, etc.
    public ValueTask EnqueueAsync(string payload, CancellationToken ct = default)
        => _workQueue.Writer.WriteAsync(payload, ct);

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("KernelWorker started. Polling every {Interval}s", _pollInterval.TotalSeconds);

        // Seed some demo work so you see output immediately
        await EnqueueAsync("I need help with my billing invoice");
        await EnqueueAsync("Want to place a new order for 50 units");
        await EnqueueAsync("Can you call me tomorrow about the refund?");

        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                // Wait for next item with cancellation support
                var payload = await _workQueue.Reader.ReadAsync(stoppingToken);
                await ProcessMessageAsync(payload, stoppingToken);
            }
            catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
            {
                // Expected on shutdown
                break;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Unexpected error in worker loop");
                // Brief backoff to avoid tight error loops
                await Task.Delay(TimeSpan.FromSeconds(10), stoppingToken);
            }
        }

        _logger.LogInformation("KernelWorker stopping");
    }

    private async Task ProcessMessageAsync(string message, CancellationToken ct)
    {
        _logger.LogInformation("Processing message: {Message}", message);

        // Invoke the kernel with the classification plugin
        var kernelArguments = new KernelArguments
        {
            ["message"] = message,
            ["categories"] = "sales,support,billing,general"
        };

        // Run the classify_intent function
        var classifyResult = await _kernel.InvokeAsync(
            "ClassificationPlugin", "classify_intent", kernelArguments, ct);

        var intent = classifyResult.GetValue<string>()?.Trim() ?? "unknown";

        // Run extract_actions
        var actionsResult = await _kernel.InvokeAsync(
            "ClassificationPlugin", "extract_actions", kernelArguments, ct);

        var actionsJson = actionsResult.GetValue<string>() ?? "[]";

        _logger.LogInformation("Intent: {Intent}, Actions: {Actions}", intent, actionsJson);

        // Here you would persist results, publish events, call downstream APIs, etc.
        // For demo we just log.
    }
}

Checkpoint: Run dotnet run. You should see output similar to:

info: SkBackgroundWorker.Services.KernelWorker[0]
      KernelWorker started. Polling every 5s
info: SkBackgroundWorker.Services.KernelWorker[0]
      Processing message: I need help with my billing invoice
info: SkBackgroundWorker.Services.KernelWorker[0]
      Intent: billing, Actions: []
info: SkBackgroundWorker.Services.KernelWorker[0]
      Processing message: Want to place a new order for 50 units
info: SkBackgroundWorker.Services.KernelWorker[0]
      Intent: sales, Actions: []
info: SkBackgroundWorker.Services.KernelWorker[0]
      Processing message: Can you call me tomorrow about the refund?
info: SkBackgroundWorker.Services.KernelWorker[0]
      Intent: support, Actions: ["schedule_call"]

The service processes the seeded messages, classifies them, and extracts actions. The plugin stubs keep this deterministic; swap them for prompt functions when you connect a real model.

Swap stubs for prompt functions

For production, replace the C# stubs with Semantic Kernel prompt functions that call the LLM. Create Plugins/ClassificationPrompts.yaml:

schema: 1
name: ClassificationPlugin
functions:
  classify_intent:
    parameters:
      - name: message
        type: string
      - name: categories
        type: string
    prompt: |
      Classify the following message into ONE of these categories: {{categories}}

      Message: "{{message}}"

      Respond with ONLY the category name.
  extract_actions:
    parameters:
      - name: message
        type: string
    prompt: |
      Extract actionable items from this message as a JSON array of strings.
      Use these action types: schedule_call, send_email, process_refund, create_ticket, escalate.

      Message: "{{message}}"

      Respond with ONLY the JSON array.

Load the prompt functions in Program.cs (replace the AddKernelPlugin<ClassificationPlugin>() line):

// Load prompt functions from YAML
var plugin = builder.Kernel.ImportPluginFromYaml(
    pluginName: "ClassificationPlugin",
    yamlPath: "Plugins/ClassificationPrompts.yaml");

Remove the ClassificationPlugin class registration and the ClassificationPlugin constructor parameter from KernelWorker. The kernel now invokes the LLM for each function. Update ProcessMessageAsync to pass the same arguments — the function signatures match.

Checkpoint: Run again with a valid API key. Logs now show LLM-driven classifications and richer action arrays:

info: SkBackgroundWorker.Services.KernelWorker[0]
      Processing message: Can you call me tomorrow about the refund?
info: SkBackgroundWorker.Services.KernelWorker[0]
      Intent: billing, Actions: ["schedule_call", "process_refund"]

Add resilience: retries, timeouts, and circuit breaking

Background services must survive transient failures. Wrap kernel invocations with Polly policies. Add the package:

dotnet add package Microsoft.Extensions.Http.Polly

Create a resilience pipeline in Program.cs:

using Microsoft.Extensions.Http.Resilience;

builder.Services.AddResiliencePipeline("kernel", pipeline =>
{
    pipeline.AddRetry(new HttpRetryStrategyOptions
    {
        MaxRetryAttempts = 3,
        Delay = TimeSpan.FromSeconds(2),
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true,
        ShouldHandle = new PredicateBuilder().Handle<HttpRequestException>()
    });
    pipeline.AddTimeout(TimeSpan.FromSeconds(30));
    pipeline.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
    {
        FailureRatio = 0.5,
        MinimumThroughput = 10,
        SamplingDuration = TimeSpan.FromMinutes(1),
        BreakDuration = TimeSpan.FromSeconds(30)
    });
});

Inject ResiliencePipelineProvider<string> into KernelWorker and wrap each InvokeAsync:

private readonly ResiliencePipeline _pipeline;

public KernelWorker(IKernel kernel, ILogger<KernelWorker> logger,
    ResiliencePipelineProvider<string> pipelineProvider)
{
    _kernel = kernel;
    _logger = logger;
    _pipeline = pipelineProvider.GetPipeline("kernel");
}

private async Task<T> InvokeWithResilience<T>(Func<Task<T>> action, CancellationToken ct)
    => await _pipeline.ExecuteAsync(async token => await action(), ct);

Then in ProcessMessageAsync:

var classifyResult = await InvokeWithResilience(
    () => _kernel.InvokeAsync("ClassificationPlugin", "classify_intent", kernelArguments, ct), ct);

This ensures a flaky provider doesn’t crash the worker or block the queue indefinitely.

Structured logging and observability

Replace the ad-hoc log lines with structured fields so your log aggregator (Seq, Datadog, Elastic) can query them. Install Serilog.AspNetCore and configure in Program.cs:

using Serilog;

builder.Services.AddSerilog((services, config) => config
    .ReadFrom.Configuration(builder.Configuration)
    .ReadFrom.Services(services)
    .Enrich.FromLogContext()
    .WriteTo.Console());

Update KernelWorker to use LogContext:

using Serilog.Context;

private async Task ProcessMessageAsync(string message, CancellationToken ct)
{
    using (LogContext.PushProperty("MessageId", Guid.NewGuid()))
    using (LogContext.PushProperty("MessageLength", message.Length))
    {
        _logger.LogInformation("Processing message");
        // ... existing logic ...
        _logger.LogInformation("Completed: Intent={Intent}, ActionCount={ActionCount}",
            intent, JsonSerializer.Deserialize<string[]>(actionsJson)?.Length ?? 0);
    }
}

Now every log entry carries correlation IDs and measurable dimensions.

Graceful shutdown and in-flight completion

BackgroundService gives you a CancellationToken that fires on SIGTERM/Ctrl+C. The Channel.Reader.ReadAsync respects it, but in-flight work may need more time. Override StopAsync to wait for the current item:

private Task? _currentWork;
private readonly SemaphoreSlim _workGate = new(1, 1);

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    // ... existing loop ...
    while (!stoppingToken.IsCancellationRequested)
    {
        var payload = await _workQueue.Reader.ReadAsync(stoppingToken);
        await _workGate.WaitAsync(stoppingToken);
        try
        {
            _currentWork = ProcessMessageAsync(payload, stoppingToken);
            await _currentWork;
        }
        finally
        {
            _currentWork = null;
            _workGate.Release();
        }
    }
}

public override async Task StopAsync(CancellationToken cancellationToken)
{
    _logger.LogInformation("StopAsync called, waiting for in-flight work");
    await _workGate.WaitAsync(TimeSpan.FromSeconds(30), cancellationToken);
    try
    {
        if (_currentWork != null)
        {
            await Task.WhenAny(_currentWork, Task.Delay(TimeSpan.FromSeconds(25), cancellationToken));
        }
    }
    finally
    {
        _workGate.Release();
    }
    await base.StopAsync(cancellationToken);
}

This gives the current message up to 30 seconds to finish before the process exits — critical for Kubernetes preStop hooks.

Testing the worker locally

You can exercise the full pipeline without deploying. Create a simple console test or use the HTTP endpoint pattern. Add a minimal API to enqueue work:

dotnet add package Microsoft.AspNetCore.OpenApi

Update Program.cs to also build a web host (dual host pattern):

// ... after builder.Services.AddHostedService<KernelWorker>()

// Also expose an HTTP endpoint for manual testing
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var host = builder.Build();

if (host.Environment.IsDevelopment())
{
    host.UseSwagger();
    host.UseSwaggerUI();
}

host.MapPost("/enqueue", async (KernelWorker worker, EnqueueRequest req, CancellationToken ct) =>
{
    await worker.EnqueueAsync(req.Message, ct);
    return Results.Accepted();
})
.WithName("EnqueueWork")
.WithOpenApi();

host.Run();

record EnqueueRequest(string Message);

Run dotnet run, open http://localhost:5000/swagger, and POST to /enqueue with { "message": "I need a refund for order 123" }. Watch the structured logs appear in your console.

Production deployment notes

• Configuration: Use Azure Key Vault, AWS Secrets Manager, or Kubernetes secrets for the API key. Never commit appsettings.Production.json.
• Scaling: Run multiple replicas behind a queue (Azure Service Bus, RabbitMQ, Kafka). Replace the in-memory Channel with a durable consumer.
• Idempotency: Include a MessageId in each payload and track processed IDs in Redis or a database to avoid duplicate work on retries.
• Model routing: If you use multiple models (e.g., a cheap classifier and an expensive extractor), configure separate serviceId values in AddOpenAIChatCompletion and reference them in the prompt function metadata.
• Metering: Hook FunctionInvoked events on the kernel to emit token usage to your observability stack.

If you operate across multiple providers, a gateway that normalizes the OpenAI contract and handles fallback can simplify the client code. n4n.ai exposes one endpoint for 240+ models and forwards provider cache-control hints, so your background service sees a stable interface even when upstream capacity shifts.

Summary

You now have a Semantic Kernel .NET background service that:

• Registers the kernel and plugins through DI
• Processes queued work with cancellation support
• Invokes LLM prompt functions via the kernel
• Applies retry, timeout, and circuit-breaker policies
• Emits structured, queryable logs
• Shuts down gracefully with in-flight completion

The pattern scales: swap the in-memory channel for a durable queue, add more plugins, and deploy multiple replicas. The kernel stays a singleton per process, plugins stay testable, and the hosting model remains standard .NET.