Semantic Kernel .NET tutorial: on-prem gateway setup

If you’re running Semantic Kernel in a regulated environment or just want full control over model routing, wiring it to a semantic kernel .net on-prem n4n.ai gateway is the most direct path. The gateway speaks the OpenAI API contract, so the Semantic Kernel OpenAI connector works without custom middleware. This tutorial walks through provisioning the gateway, configuring the kernel, and validating the loop with a minimal chat completion call.

Prerequisites

• .NET 8 SDK installed
• Access to a running n4n.ai gateway instance (self-hosted or managed) with at least one model deployed
• An API key issued by the gateway for your service account
• Network connectivity from your application host to the gateway endpoint (default port 4000 for HTTP, 4443 for HTTPS)

If you’re spinning up the gateway yourself, the quickstart container image exposes the OpenAI-compatible endpoint at http://<host>:4000/v1. Verify it responds before proceeding:

curl -s http://localhost:4000/v1/models | jq '.data[].id'

You should see a list of model identifiers the gateway serves.

Step 1: Create the project and add packages

Start a new console app or add to an existing solution. You need the core Semantic Kernel package plus the OpenAI connector.

dotnet new console -n SkOnPremDemo
cd SkOnPremDemo
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.OpenAI

The OpenAI connector is the bridge — it targets the OpenAI REST surface, which the gateway implements.

Step 2: Configure the kernel builder

Open Program.cs and replace the template with a kernel builder pointed at your gateway. The critical pieces are the endpoint base address and the API key.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;

var gatewayEndpoint = "http://gateway.internal:4000/v1"; // adjust host/port
var apiKey = "sk-gateway-..."; // your service account key
var deploymentName = "llama-3-70b-instruct"; // model id as known by the gateway

var builder = Kernel.CreateBuilder();

// Add the OpenAI-compatible chat completion service
builder.AddOpenAIChatCompletion(
    modelId: deploymentName,
    apiKey: apiKey,
    endpoint: new Uri(gatewayEndpoint),
    serviceId: "on-prem-gateway" // optional, useful for multi-service kernels
);

var kernel = builder.Build();

Notes:

• modelId must match exactly what the gateway returns from /v1/models. Case-sensitive.
• serviceId lets you address this specific service later if you register multiple connectors (for example, a local GGUF model alongside the gateway).
• The endpoint must include the /v1 path prefix; the connector appends /chat/completions automatically.

Step 3: Execute a chat completion

Add a simple invocation to verify the pipeline works end to end.

var chat = kernel.GetRequiredService<IChatCompletionService>();

var history = new ChatHistory();
history.AddSystemMessage("You are a concise assistant running on-prem.");
history.AddUserMessage("Explain the difference between a mutex and a semaphore in two sentences.");

var response = await chat.GetChatMessageContentAsync(history);
Console.WriteLine(response.Content);

Run it:

dotnet run

You should see a concise answer printed to stdout. If you get a 404, double-check the modelId spelling against /v1/models. If you get a 401, confirm the API key is valid and not expired. A 502 or 503 usually means the gateway couldn’t reach an upstream provider — check the gateway logs.

Step 4: Add structured logging and retries

Production code needs observability and resilience. Semantic Kernel uses Microsoft.Extensions.Logging; wire a console logger and configure a retry policy on the HTTP client.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Polly;
using Polly.Extensions.Http;

var services = new ServiceCollection();
services.AddLogging(b => b.AddConsole().SetMinimumLevel(LogLevel.Information));

// Retry on transient failures (5xx, 429, network errors)
services.AddHttpClient("gateway")
    .AddPolicyHandler(HttpPolicyExtensions
        .HandleTransientHttpError()
        .OrResult(r => r.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
        .WaitAndRetryAsync(3, attempt => TimeSpan.FromSeconds(Math.Pow(2, attempt))));

var builder = Kernel.CreateBuilder(services);

builder.AddOpenAIChatCompletion(
    modelId: deploymentName,
    apiKey: apiKey,
    endpoint: new Uri(gatewayEndpoint),
    serviceId: "on-prem-gateway",
    httpClientName: "gateway" // binds the typed client with retry policy
);

var kernel = builder.Build();
var chat = kernel.GetRequiredService<IChatCompletionService>();

// ... same invocation as Step 3

The httpClientName parameter tells the connector to pull the named HttpClient from DI, giving you centralized retry, timeout, and header logic.

Step 5: Enable streaming for lower perceived latency

Streaming returns tokens as they arrive, which matters for UX in chat interfaces. The connector supports IAsyncEnumerable<StreamingChatMessageContent>.

var streamingChat = kernel.GetRequiredService<IChatCompletionService>();

await foreach (var chunk in streamingChat.GetStreamingChatMessageContentsAsync(history))
{
    Console.Write(chunk.Content);
}
Console.WriteLine();

Each chunk contains a token or partial token. Assemble them if you need the full response for downstream processing.

Step 6: Use kernel functions for prompt templates

Semantic Kernel’s value shows up when you parameterize prompts as functions. Define a YAML prompt file (Prompts/Summarize.yaml):

name: Summarize
template: |
  Summarize the following text in {{style}} style, max 3 sentences.

  Text:
  {{$input}}
template_format: semantic-kernel
input_variables:
  - name: input
    is_required: true
  - name: style
    is_required: true
    default: "executive"

Load and invoke it:

var plugin = kernel.ImportPluginFromPromptDirectory("Prompts");
var summarize = plugin["Summarize"];

var result = await kernel.InvokeAsync(summarize, new()
{
    ["input"] = File.ReadAllText("long-report.txt"),
    ["style"] = "technical"
});

Console.WriteLine(result.GetValue<string>());

The prompt runs against the gateway-backed model. You can swap the model by changing deploymentName without touching prompt code.

Step 7: Health checks and readiness probes

If you’re deploying to Kubernetes or a similar orchestrator, expose a health endpoint that verifies the gateway is reachable and the model responds.

app.MapGet("/health/ready", async (Kernel kernel) =>
{
    try
    {
        var chat = kernel.GetRequiredService<IChatCompletionService>();
        var test = new ChatHistory();
        test.AddUserMessage("ping");
        await chat.GetChatMessageContentAsync(test, kernel: kernel, cancellationToken: CancellationToken.None);
        return Results.Ok(new { status = "ready", gateway = gatewayEndpoint });
    }
    catch (Exception ex)
    {
        return Results.Problem(
            detail: ex.Message,
            statusCode: 503,
            title: "Gateway unavailable"
        );
    }
});

This does a real (tiny) inference call. For a lighter check, the gateway itself exposes /healthz — call that instead if you only need to know the gateway process is up.

Step 8: Observability — metrics and tracing

The gateway emits OpenTelemetry metrics by default (request latency, token counts, error rates). Wire the .NET side to propagate trace context so you can correlate a kernel invocation across your service and the gateway.

using System.Diagnostics;
using OpenTelemetry.Trace;

var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSource("Microsoft.SemanticKernel*")
    .AddHttpClientInstrumentation()
    .AddOtlpExporter()
    .Build();

// In your kernel builder, ensure HttpClient instrumentation captures the gateway calls
services.AddHttpClient("gateway")
    .AddPolicyHandler(...)
    .ConfigureHttpClient(c => c.DefaultRequestHeaders.Add("X-Request-ID", Activity.Current?.Id ?? Guid.NewGuid().ToString()));

The X-Request-ID header lets you stitch logs from your app to the gateway’s access logs. The gateway also forwards provider cache-control hints (e.g., x-cache-hit: true) — log those to measure cache effectiveness.

Step 9: Multi-model routing with service IDs

Register multiple chat completion services pointing at different gateway model deployments, then select at runtime.

builder.AddOpenAIChatCompletion(
    modelId: "llama-3-70b-instruct",
    apiKey: apiKey,
    endpoint: new Uri(gatewayEndpoint),
    serviceId: "large"
);

builder.AddOpenAIChatCompletion(
    modelId: "llama-3-8b-instruct",
    apiKey: apiKey,
    endpoint: new Uri(gatewayEndpoint),
    serviceId: "small"
);

var kernel = builder.Build();

// Choose per request
var largeChat = kernel.GetRequiredService<IChatCompletionService>("large");
var smallChat = kernel.GetRequiredService<IChatCompletionService>("small");

This pattern lets you route simple classification tasks to the small model and complex reasoning to the large one, all through the same gateway endpoint.

Step 10: Verify the full loop with a scripted test

Create a small integration test that runs in CI against a staging gateway. This catches drift between your model IDs and what the gateway actually serves.

// Tests/Integration/GatewayIntegrationTests.cs
[Fact]
public async Task Gateway_ChatCompletion_ReturnsResponse()
{
    var kernel = Kernel.CreateBuilder()
        .AddOpenAIChatCompletion(
            modelId: "llama-3-70b-instruct",
            apiKey: Environment.GetEnvironmentVariable("GATEWAY_API_KEY")!,
            endpoint: new Uri(Environment.GetEnvironmentVariable("GATEWAY_ENDPOINT")!))
        .Build();

    var chat = kernel.GetRequiredService<IChatCompletionService>();
    var history = new ChatHistory();
    history.AddUserMessage("Reply with the single word: pong");

    var result = await chat.GetChatMessageContentAsync(history);
    
    Assert.Contains("pong", result.Content!, StringComparison.OrdinalIgnoreCase);
}

Run with dotnet test --filter "FullyQualifiedName~GatewayIntegrationTests". Set the env vars in your CI pipeline. This test validates connectivity, auth, model ID, and basic inference in one shot.

Common pitfalls

What’s next

You now have a Semantic Kernel application talking to an on-prem gateway with retries, streaming, structured prompts, health checks, and multi-model routing. From here you can:

• Add vector store connectors (Qdrant, Weaviate, PGVector) for RAG pipelines
• Implement planners for multi-step agent workflows
• Enforce policy with kernel filters (PII redaction, token budgets)
• Hook the gateway’s per-token metering into your cost allocation system

The gateway handles provider diversity, fallback, and usage accounting; your code stays focused on prompt engineering and business logic.