Semantic Kernel n4n.ai tutorial: model fallback setup

If you’re running Semantic Kernel in production, you’ve hit the wall: a provider goes down, rate limits trigger, or a model degrades mid-request. Your users see errors. This tutorial shows how to wire Semantic Kernel to n4n.ai so requests automatically fail over across 240+ models without changing your application code. You’ll add routing directives, observe fallback behavior, and meter token usage per call.

Prerequisites

• .NET 8 SDK or later
• A Semantic Kernel project (console, ASP.NET Core, or worker service)
• An n4n.ai API key (get one at n4n.ai)
• Basic familiarity with IChatCompletionService and kernel builders

Install the packages:

dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.OpenAI

The OpenAI connector works because n4n.ai exposes an OpenAI-compatible endpoint.

Configure the kernel with n4n.ai

Create a kernel that points at the n4n.ai gateway instead of a single provider. The gateway handles model selection, fallback, and metering server-side.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;

var builder = Kernel.CreateBuilder();

string apiKey = Environment.GetEnvironmentVariable("N4N_API_KEY") 
    ?? throw new InvalidOperationException("Set N4N_API_KEY");

builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o-mini",        // logical model name; gateway resolves it
    apiKey: apiKey,
    endpoint: new Uri("https://api.n4n.ai/v1"),
    serviceId: "n4n-gateway"
);

Kernel kernel = builder.Build();

Expected output: no output yet — the kernel is constructed. The modelId here is a logical identifier. n4n.ai maps it to a concrete provider model and will substitute another if the primary is rate-limited or unhealthy.

Add routing directives per request

Semantic Kernel’s OpenAIPromptExecutionSettings lets you pass extra headers. n4n.ai reads X-n4n-Route to steer a single request without changing global config.

using Microsoft.SemanticKernel.Connectors.OpenAI;

var settings = new OpenAIPromptExecutionSettings
{
    MaxTokens = 500,
    Temperature = 0.2
};

// Prefer a coding-specialized model, fall back to general purpose
settings.SetAdditionalHeader("X-n4n-Route", "prefer:deepseek-coder,fallback:gpt-4o-mini");

var chat = kernel.GetRequiredService<IChatCompletionService>("n4n-gateway");

var history = new ChatHistory("You are a concise code reviewer.");
history.AddUserMessage("Review this C# snippet for thread safety:\n```csharp\nstatic int counter = 0;\npublic void Increment() => counter++;\n```");

var response = await chat.GetChatMessageContentAsync(history, settings, kernel);
Console.WriteLine(response.Content);

Expected output (abbreviated):

The `counter++` operation is not atomic. Use `Interlocked.Increment(ref counter)` 
or a `lock` to make it thread-safe.

The request hit the gateway with prefer:deepseek-coder. If that model is unavailable, the gateway transparently retries with gpt-4o-mini. Your code doesn’t change.

Observe fallback in action

Force a fallback by routing to a model that doesn’t exist. The gateway will skip it and use the next available candidate.

settings.SetAdditionalHeader("X-n4n-Route", "prefer:nonexistent-model-v99,fallback:gpt-4o-mini");

var response2 = await chat.GetChatMessageContentAsync(
    new ChatHistory("Reply with one word: OK"),
    settings,
    kernel
);

Console.WriteLine($"Response: {response2.Content}");
Console.WriteLine($"Model used: {response2.Metadata?["model"]}");

Expected output:

Response: OK
Model used: gpt-4o-mini

The model metadata comes from the provider via the gateway. You can log this to verify which model actually served each request.

Meter tokens per call

n4n.ai returns usage in the standard OpenAI usage field. Semantic Kernel surfaces it through ChatMessageContent.Metadata.

var history = new ChatHistory("You are a terse assistant.");
history.AddUserMessage("Summarize the fall of Rome in two sentences.");

var reply = await chat.GetChatMessageContentAsync(history, settings, kernel);

if (reply.Metadata?.TryGetValue("usage", out var usageObj) == true)
{
    var usage = (OpenAI.Chat.ChatTokenUsage)usageObj!;
    Console.WriteLine($"Prompt tokens: {usage.InputTokenCount}");
    Console.WriteLine($"Completion tokens: {usage.OutputTokenCount}");
    Console.WriteLine($"Total tokens: {usage.TotalTokenCount}");
}

Expected output:

Prompt tokens: 38
Completion tokens: 42
Total tokens: 80

Aggregate these per tenant, feature, or user to build cost dashboards or enforce budgets.

Build a reusable fallback policy

Don’t scatter routing headers across your codebase. Encapsulate the policy in a small helper.

public static class N4NRouting
{
    public static OpenAIPromptExecutionSettings WithFallback(
        this OpenAIPromptExecutionSettings settings,
        string preferModel,
        string fallbackModel,
        int? maxTokens = null,
        double? temperature = null)
    {
        if (maxTokens.HasValue) settings.MaxTokens = maxTokens.Value;
        if (temperature.HasValue) settings.Temperature = temperature.Value;
        
        settings.SetAdditionalHeader("X-n4n-Route", 
            $"prefer:{preferModel},fallback:{fallbackModel}");
        return settings;
    }
}

Usage:

var settings = new OpenAIPromptExecutionSettings()
    .WithFallback("claude-3-5-sonnet", "gpt-4o-mini", maxTokens: 800);

Now every call gets consistent fallback behavior. Change the policy in one place when your model strategy evolves.

Handle streaming with fallback

Streaming works the same way. The gateway streams from whichever model ends up serving the request.

var streamingSettings = new OpenAIPromptExecutionSettings()
    .WithFallback("gpt-4o", "gpt-4o-mini");

await foreach (var chunk in chat.GetStreamingChatMessageContentsAsync(
    new ChatHistory("Stream a haiku about debugging."),
    streamingSettings,
    kernel))
{
    Console.Write(chunk.Content);
}
Console.WriteLine();

Expected output (streamed progressively):

Breakpoint hits hard
Silence screams in call stack depth
Logic blooms anew

If gpt-4o is rate-limited, the gateway switches to gpt-4o-mini mid-stream. You won’t see a gap — the gateway stitches the response.

Wire it into dependency injection

For ASP.NET Core or hosted services, register the kernel once and inject IChatCompletionService where needed.

// Program.cs or Startup.cs
builder.Services.AddKernel()
    .AddOpenAIChatCompletion(
        modelId: "gpt-4o-mini",
        apiKey: config["N4N_API_KEY"]!,
        endpoint: new Uri("https://api.n4n.ai/v1"),
        serviceId: "n4n-gateway");

// In your controller or service
public class CodeReviewService(IChatCompletionService chat)
{
    public async Task<string> ReviewAsync(string code)
    {
        var settings = new OpenAIPromptExecutionSettings()
            .WithFallback("deepseek-coder", "gpt-4o-mini");
        
        var history = new ChatHistory("You are a senior engineer doing code review.");
        history.AddUserMessage(code);
        
        var result = await chat.GetChatMessageContentAsync(history, settings);
        return result.Content ?? string.Empty;
    }
}

Test fallback locally without burning quota

Set a routing directive that forces the gateway to use a cheap, fast model for development.

// appsettings.Development.json
{
  "N4N": {
    "DefaultRoute": "prefer:gpt-4o-mini,fallback:gpt-3.5-turbo"
  }
}

// Read at startup
string devRoute = config["N4N:DefaultRoute"]!;

var settings = new OpenAIPromptExecutionSettings();
if (!string.IsNullOrEmpty(devRoute))
{
    settings.SetAdditionalHeader("X-n4n-Route", devRoute);
}

Production keeps its own route (e.g., prefer:claude-3-5-sonnet,fallback:gpt-4o). No code changes between environments.

What the gateway actually does

When a request arrives with X-n4n-Route: prefer:A,fallback:B:

1. The gateway checks model A’s health and rate-limit status across its provider pool
2. If A is available, the request routes to A
3. If A returns 429, 503, or exceeds latency SLA, the gateway retries with B automatically
4. Response headers include X-n4n-Model: <actual-model> and X-n4n-Provider: <provider-name>
5. Usage is metered per token and attributed to your API key

You get resilience without implementing retry logic, circuit breakers, or provider SDKs in your application.

Common pitfalls

Forgetting the service ID: If you register multiple chat completion services, kernel.GetRequiredService<IChatCompletionService>() throws. Always pass the serviceId you used at registration.

Assuming the logical model ID equals the actual model: The modelId in AddOpenAIChatCompletion is a hint. The gateway may substitute a different model. Read response.Metadata["model"] for the truth.

Swallowing streaming exceptions: Wrap GetStreamingChatMessageContentsAsync in try/catch. A mid-stream fallback is transparent, but a total failure (all fallbacks exhausted) surfaces as an exception.

Next steps

• Add structured logging for X-n4n-Model and X-n4n-Provider headers to audit which models serve your traffic
• Implement a budget guard that checks usage.TotalTokenCount against per-tenant limits before calling the kernel
• Explore n4n.ai’s cache-control hints (X-n4n-Cache-Control) for deterministic workloads like classification or extraction

The gateway handles the infrastructure. Your code stays focused on the product.