Semantic Kernel .NET tutorial: retries and timeouts

If you’re building production systems with Semantic Kernel, you need a semantic kernel .net retries timeouts tutorial that shows exactly how to handle transient failures without drowning in boilerplate. The kernel’s HTTP pipeline sits on HttpClient, which means every provider call — OpenAI, Azure OpenAI, Anthropic, local models — inherits the same reliability gaps: no retries by default, no timeout enforcement, and no circuit breaking. This post walks through wiring Polly policies into the kernel’s HttpClient factory so your agents survive rate limits, network blips, and stuck requests.

Prerequisites

.NET 8 SDK or later
A Semantic Kernel project targeting net8.0
NuGet packages: Microsoft.SemanticKernel, Microsoft.Extensions.Http.Polly, Polly.Extensions.Http
An API key for at least one provider (OpenAI, Azure OpenAI, etc.)

Create a new console app if you’re starting fresh:

dotnet new console -n SKReliabilityDemo
cd SKReliabilityDemo
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.Extensions.Http.Polly
dotnet add package Polly.Extensions.Http

The default behavior is dangerous

Out of the box, Kernel.CreateBuilder().AddOpenAIChatCompletion(...).Build() hands you an HttpClient with infinite timeout and zero retry logic. A single hung socket or 429 response blocks your thread indefinitely. Let’s prove it.

// Program.cs - baseline (do not ship this)
using Microsoft.SemanticKernel;

var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o-mini",
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")!);

var kernel = builder.Build();

var result = await kernel.InvokePromptAsync("Say hello in one sentence.");
Console.WriteLine(result);

Run it. It works — until it doesn’t. Now let’s fix it properly.

Configure retries with Polly

Semantic Kernel uses IHttpClientFactory under the hood. You attach policies by calling AddHttpClient on the builder’s service collection before building the kernel.

// Program.cs - with retry policy
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;
using Microsoft.SemanticKernel;
using Polly;
using Polly.Extensions.Http;

var builder = Kernel.CreateBuilder();

// Register the retry policy on the named HttpClient SK uses
builder.Services.AddHttpClient("SemanticKernel", client =>
{
    client.Timeout = TimeSpan.FromSeconds(100); // upper bound for the whole request
})
.AddStandardResilienceHandler(options =>
{
    // Retry on transient failures: 429, 5xx, timeouts, network errors
    options.Retry.MaxRetryAttempts = 3;
    options.Retry.Delay = TimeSpan.FromSeconds(2);
    options.Retry.BackoffType = DelayBackoffType.Exponential;
    options.Retry.UseJitter = true;
    options.Retry.ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
        .HandleResult(r => (int)r.StatusCode == 429) // rate limited
        .HandleResult(r => (int)r.StatusCode >= 500) // server errors
        .Handle<TimeoutRejectedException>()          // Polly timeout
        .Handle<HttpRequestException>();             // network errors
});

builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o-mini",
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")!,
    httpClientName: "SemanticKernel"); // critical: bind to our configured client

var kernel = builder.Build();

var result = await kernel.InvokePromptAsync("Say hello in one sentence.");
Console.WriteLine(result);

Expected output (success case):

Hello! How can I help you today?

What changed: AddStandardResilienceHandler (from Microsoft.Extensions.Http.Resilience) wraps Polly’s StandardResiliencePipeline with sensible defaults. The ShouldHandle predicate ensures we retry on 429, 5xx, timeouts, and network-level exceptions. Exponential backoff with jitter prevents thundering herds when a provider recovers.

Separate timeout policies for connect vs. request

A single HttpClient.Timeout covers the entire request lifecycle — DNS, TLS handshake, request send, response read. For LLM streaming, you often want a tight connect timeout but a generous read timeout. Split them with PooledConnectionLifetime and per-request TimeoutPolicy.

// Program.cs - granular timeouts
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;
using Microsoft.SemanticKernel;
using Polly;
using Polly.Timeout;

var builder = Kernel.CreateBuilder();

builder.Services.AddHttpClient("SemanticKernel", client =>
{
    // Connection-level timeouts
    client.DefaultRequestHeaders.ConnectionClose = false; // keep-alive
})
.ConfigurePrimaryHttpMessageHandler(() => new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(2),
    PooledConnectionIdleTimeout = TimeSpan.FromMinutes(1),
    ConnectTimeout = TimeSpan.FromSeconds(5), // TCP + TLS handshake
    EnableMultipleHttp2Connections = true,
})
.AddStandardResilienceHandler(options =>
{
    // Total request timeout (connect + send + read)
    options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(120);
    
    // Retry config from previous section
    options.Retry.MaxRetryAttempts = 3;
    options.Retry.Delay = TimeSpan.FromSeconds(2);
    options.Retry.BackoffType = DelayBackoffType.Exponential;
    options.Retry.UseJitter = true;
    options.Retry.ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
        .HandleResult(r => (int)r.StatusCode == 429)
        .HandleResult(r => (int)r.StatusCode >= 500)
        .Handle<TimeoutRejectedException>()
        .Handle<HttpRequestException>();
    
    // Circuit breaker: stop hammering a down provider
    options.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(30);
    options.CircuitBreaker.MinimumThroughput = 5;
    options.CircuitBreaker.FailureRatio = 0.5;
    options.CircuitBreaker.BreakDuration = TimeSpan.FromSeconds(30);
});

builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o-mini",
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")!,
    httpClientName: "SemanticKernel");

var kernel = builder.Build();

// Test streaming with the new timeouts
await foreach (var chunk in kernel.InvokePromptStreamingAsync("Count to 10 slowly."))
{
    Console.Write(chunk);
}
Console.WriteLine();

Expected output (streaming):

1, 2, 3, 4, 5, 6, 7, 8, 9, 10.

The SocketsHttpHandler configuration gives you control over connection pooling and the TCP connect timeout. TotalRequestTimeout in the resilience handler caps the full request including streaming. The circuit breaker trips after 50% failures over 30 seconds with at least 5 requests, then holds open for 30 seconds — preventing cascade failures when a provider goes dark.

Per-request overrides for long-running operations

Some prompts need more time (large context, reasoning models). Override the pipeline per-invocation without changing global config.

// Program.cs - per-request timeout override
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;
using Microsoft.SemanticKernel;
using Polly;

var builder = Kernel.CreateBuilder();

// ... same HttpClient registration as above ...

var kernel = builder.Build();

// Short timeout for simple classification
var shortResult = await kernel.InvokePromptAsync(
    "Classify: 'I love this product!' -> positive/negative",
    new KernelArguments { { "timeout", TimeSpan.FromSeconds(10) } });

// Long timeout for complex reasoning
var longResult = await kernel.InvokePromptAsync(
    "Write a detailed technical specification for a distributed cache.",
    new KernelArguments { { "timeout", TimeSpan.FromMinutes(5) } });

Semantic Kernel doesn’t natively read a timeout argument. You implement this by creating a custom DelegatingHandler that reads KernelArguments and sets HttpRequestMessage.Options.Set(new HttpRequestOptionsKey<TimeSpan>("timeout"), value). Here’s the minimal implementation:

// PerRequestTimeoutHandler.cs
using Microsoft.Extensions.Http.Resilience;
using Microsoft.SemanticKernel;
using System.Net.Http;

public class PerRequestTimeoutHandler : DelegatingHandler
{
    private readonly IServiceProvider _services;

    public PerRequestTimeoutHandler(IServiceProvider services)
    {
        _services = services;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (request.Options.TryGetValue(new HttpRequestOptionsKey<TimeSpan>("timeout"), out var timeout))
        {
            // Create a linked token that respects the per-request timeout
            using var timeoutCts = new CancellationTokenSource(timeout);
            using var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(
                cancellationToken, timeoutCts.Token);
            
            return await base.SendAsync(request, linkedCts.Token);
        }
        return await base.SendAsync(request, cancellationToken);
    }
}

builder.Services.AddHttpClient("SemanticKernel", client => { })
    .AddHttpMessageHandler<PerRequestTimeoutHandler>()
    .AddStandardResilienceHandler(options => { /* ... */ });

builder.Services.AddTransient<PerRequestTimeoutHandler>();

Now inject the timeout via KernelArguments in your prompt invocation code (requires a small middleware or filter to move the argument into HttpRequestMessage.Options — see the SK docs for IFunctionInvocationFilter).

Observability: log every retry and timeout

You can’t tune what you can’t see. Hook Polly’s onRetry and onTimeout callbacks to emit structured logs.

// Program.cs - with logging
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Http.Resilience;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Polly;

var builder = Kernel.CreateBuilder();

builder.Logging.AddConsole().SetMinimumLevel(LogLevel.Debug);

builder.Services.AddHttpClient("SemanticKernel", client => { })
    .AddStandardResilienceHandler(options =>
    {
        options.Retry.MaxRetryAttempts = 3;
        options.Retry.Delay = TimeSpan.FromSeconds(2);
        options.Retry.BackoffType = DelayBackoffType.Exponential;
        options.Retry.UseJitter = true;
        options.Retry.ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
            .HandleResult(r => (int)r.StatusCode == 429)
            .HandleResult(r => (int)r.StatusCode >= 500)
            .Handle<TimeoutRejectedException>()
            .Handle<HttpRequestException>();
        
        // Structured logging callbacks
        options.Retry.OnRetry = args =>
        {
            var logger = args.ServiceProvider.GetRequiredService<ILogger<Program>>();
            logger.LogWarning(
                "Retry {Attempt} for {Method} {Url} after {Delay}ms. Reason: {Reason}",
                args.AttemptNumber,
                args.Outcome.RequestMessage?.Method,
                args.Outcome.RequestMessage?.RequestUri,
                args.RetryDelay.TotalMilliseconds,
                args.Outcome.Exception?.Message ?? args.Outcome.Result?.StatusCode.ToString());
            return ValueTask.CompletedTask;
        };

        options.TotalRequestTimeout.OnTimeout = args =>
        {
            var logger = args.ServiceProvider.GetRequiredService<ILogger<Program>>();
            logger.LogError(
                "Request timed out after {Timeout}s: {Method} {Url}",
                args.Context.RequestMessage?.RequestUri,
                args.Timeout.TotalSeconds);
            return ValueTask.CompletedTask;
        };

        options.CircuitBreaker.OnOpened = args =>
        {
            var logger = args.ServiceProvider.GetRequiredService<ILogger<Program>>();
            logger.LogCritical("Circuit breaker OPENED for {Duration}s", args.BreakDuration.TotalSeconds);
            return ValueTask.CompletedTask;
        };
    });

builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o-mini",
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")!,
    httpClientName: "SemanticKernel");

var kernel = builder.Build();

var result = await kernel.InvokePromptAsync("Say hello.");
Console.WriteLine(result);

Expected log output (when a 429 triggers a retry):

warn: Program[0]
      Retry 1 for POST https://api.openai.com/v1/chat/completions after 2147ms. Reason: 429
info: Program[0]
      Hello! How can I help you today?

Testing the resilience locally

Don’t wait for production to verify your policies. Use a fault-injection handler or a mock server. WireMock.Net works well for this.

dotnet add package WireMock.Net

// TestResilience.cs
using WireMock.RequestBuilders;
using WireMock.ResponseBuilders;
using WireMock.Server;
using Microsoft.SemanticKernel;

var server = WireMockServer.Start();
var url = server.Urls[0];

// Simulate: 429 twice, then success
server.Given(Request.Create().WithPath("/v1/chat/completions").UsingPost())
    .InScenario("Rate limit then succeed")
    .AtPriority(1)
    .RespondWith(Response.Create().WithStatusCode(429).WithHeader("Retry-After", "1"))
    .AtPriority(2)
    .RespondWith(Response.Create().WithStatusCode(429).WithHeader("Retry-After", "1"))
    .AtPriority(3)
    .RespondWith(Response.Create()
        .WithStatusCode(200)
        .WithHeader("Content-Type", "application/json")
        .WithBody("""
        {
          "id": "test",
          "object": "chat.completion",
          "choices": [{ "index": 0, "message": { "role": "assistant", "content": "Hello from mock!" }, "finish_reason": "stop" }],
          "usage": { "prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15 }
        }
        """));

var builder = Kernel.CreateBuilder();
builder.Services.AddHttpClient("SemanticKernel", client => { })
    .AddStandardResilienceHandler(options =>
    {
        options.Retry.MaxRetryAttempts = 3;
        options.Retry.Delay = TimeSpan.FromMilliseconds(500);
        options.Retry.BackoffType = DelayBackoffType.Exponential;
        options.Retry.UseJitter = false;
        options.Retry.ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
            .HandleResult(r => (int)r.StatusCode == 429);
    });

builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o-mini",
    apiKey: "fake-key",
    endpoint: new Uri(url),
    httpClientName: "SemanticKernel");

var kernel = builder.Build();
var result = await kernel.InvokePromptAsync("Test");
Console.WriteLine($"Result: {result}");

server.Stop();

Expected output:

Result: Hello from mock!

The test proves the retry pipeline fires twice before succeeding. Adjust MaxRetryAttempts to 1 and watch it throw — confirming the policy boundary works.

Production checklist

Before shipping, verify these settings against your provider SLAs and your application’s latency budget:

Setting	Recommended starting value	Tune based on
`ConnectTimeout`	5s	Network latency to provider
`TotalRequestTimeout`	120s (streaming), 30s (non-streaming)	Model max tokens, streaming vs. blocking
`MaxRetryAttempts`	3	Cost of duplicate requests, idempotency
`BaseDelay`	2s	Provider `Retry-After` headers, rate limit window
`CircuitBreaker.FailureRatio`	0.5	Traffic volume, acceptable error rate
`CircuitBreaker.BreakDuration`	30s	Provider recovery time, fallback strategy

Idempotency matters: POST requests to /chat/completions are not inherently idempotent. If you retry a request that already succeeded but the response was lost, you’ll get duplicate completions and double billing. For critical workloads, implement request deduplication at the application layer (generate a request-id header, store results keyed by it) or use a provider that supports idempotency keys (Anthropic, some Azure OpenAI deployments).

Fallback to another model or provider

When the circuit breaker opens or retries exhaust, you need a fallback — not an exception. Semantic Kernel supports multiple IChatCompletionService registrations. Resolve them by service key and implement a fallback chain.

// Program.cs - fallback chain
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

var builder = Kernel.CreateBuilder();

// Primary: OpenAI
builder.AddOpenAIChatCompletion(
    modelId: "gpt-4o",
    apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY")!,
    serviceId: "primary");

// Fallback: Azure OpenAI (different quota pool)
builder.AddAzureOpenAIChatCompletion(
    deploymentName: "gpt-4o",
    endpoint: Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!,
    apiKey: Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY")!,
    serviceId: "fallback");

// Local fallback: Ollama (no quota, higher latency)
builder.AddOpenAIChatCompletion(
    modelId: "llama3.1",
    endpoint: new Uri("http://localhost:11434/v1"),
    apiKey: "ollama",
    serviceId: "local");

var kernel = builder.Build();

// Resolve services by key
var primary = kernel.GetRequiredService<IChatCompletionService>("primary");
var fallback = kernel.GetRequiredService<IChatCompletionService>("fallback");
var local = kernel.GetRequiredService<IChatCompletionService>("local");

async Task<string> ChatWithFallback(string prompt)
{
    try
    {
        var result = await primary.GetChatMessageContentsAsync(
            new ChatHistory(prompt), kernel: kernel);
        return result[0].Content ?? "";
    }
    catch (HttpOperationException ex) when (ex.StatusCode == System.Net.HttpStatusCode.TooManyRequests)
    {
        Console.WriteLine("Primary rate limited, trying fallback...");
        var result = await fallback.GetChatMessageContentsAsync(
            new ChatHistory(prompt), kernel: kernel);
        return result[0].Content ?? "";
    }
    catch
    {
        Console.WriteLine("Fallback failed, trying local...");
        var result = await local.GetChatMessageContentsAsync(
            new ChatHistory(prompt), kernel: kernel);
        return result[0].Content ?? "";
    }
}

Console.WriteLine(await ChatWithFallback("Hello"));

This pattern — primary cloud model, secondary cloud account, local model — covers quota exhaustion, regional outages, and complete provider downtime. Wire each service through its own HttpClient with independent resilience policies so a misconfigured timeout on the local model doesn’t affect the primary.

One endpoint, 240+ models, automatic fallback

If you’d rather not maintain multiple provider SDKs and fallback logic yourself, n4n.ai exposes a single OpenAI-compatible endpoint that routes across 240+ models and automatically fails over when a provider is rate-limited or degraded. You configure one HttpClient with the resilience policies above, point it at the n4n.ai base URL, and the gateway handles model selection, fallback, and per-token metering. The same retry and timeout code you wrote here works unchanged — the gateway just becomes the reliable upstream.

Summary

You now have a complete resilience stack for Semantic Kernel .NET:

Connection-level timeouts via SocketsHttpHandler (5s connect, pooled keep-alive)
Request-level timeout via TotalRequestTimeout (120s for streaming)
Retry policy with exponential backoff + jitter on 429, 5xx, timeouts, network errors
Circuit breaker to stop hammering a down provider (50% failure rate, 30s break)
Structured logging on every retry, timeout, and circuit state change
Per-request overrides for long-running prompts
Fallback chain across multiple IChatCompletionService registrations
Local verification with WireMock.Net

Copy the AddStandardResilienceHandler block into your DI container, tune the numbers to your latency budget, and you’ll stop losing requests to transient failures. The code is all standard .NET primitives — no framework lock-in, no magic.