Mistral Large in Semantic Kernel: a .NET integration guide

Mistral Large is Mistral AI’s flagship model, and Semantic Kernel is Microsoft’s orchestration framework for building AI applications. Getting them to work together in .NET requires navigating a few sharp edges around authentication, model naming, and the SK abstraction layer. This guide walks through a production-ready integration path.

Prerequisites and package setup

Start with a .NET 8 or later project. You need the Semantic Kernel core package plus the Mistral AI connector. The connector lives in a separate NuGet package maintained by the Semantic Kernel team.

dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.MistralAI

If you’re targeting .NET Framework 4.8.1, the packages work there too, but you’ll want .NET 8 for the best async and streaming support.

Create a MistralKernelBuilder extension to keep your composition root clean:

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.MistralAI;

public static class MistralKernelBuilder
{
    public static IKernelBuilder AddMistralLarge(
        this IKernelBuilder builder,
        string apiKey,
        string? endpoint = null)
    {
        return builder.AddMistralAIChatCompletion(
            modelId: "mistral-large-latest",
            apiKey: apiKey,
            endpoint: endpoint ?? "https://api.mistral.ai/v1");
    }
}

The model ID mistral-large-latest is an alias that Mistral updates to point at their current flagship. Pin to a specific version like mistral-large-2407 if you need reproducibility across deployments.

Basic chat completion

With the builder in place, kernel creation is straightforward. The key decision is whether to use the kernel as a service container or instantiate it per request. For web APIs, register it as a scoped service.

// Program.cs or Startup.cs
builder.Services.AddKernel()
    .AddMistralLarge(Environment.GetEnvironmentVariable("MISTRAL_API_KEY")!);

builder.Services.AddScoped<ChatService>();

public sealed class ChatService
{
    private readonly Kernel _kernel;

    public ChatService(Kernel kernel) => _kernel = kernel;

    public async Task<string> GetResponseAsync(string userMessage, CancellationToken ct = default)
    {
        var chatHistory = new ChatHistory();
        chatHistory.AddUserMessage(userMessage);

        var result = await _kernel.GetRequiredService<IChatCompletionService>()
            .GetChatMessageContentAsync(chatHistory, cancellationToken: ct);

        return result.Content ?? string.Empty;
    }
}

Pitfall: the IChatCompletionService returns a ChatMessageContent object, not a raw string. Always check Content for null — the model can return tool calls or empty responses depending on the prompt.

Streaming responses

Streaming is essential for UX. Semantic Kernel exposes GetStreamingChatMessageContentsAsync which yields StreamingChatMessageContent chunks. Aggregate them yourself if you need the full response after streaming.

public async IAsyncEnumerable<string> StreamResponseAsync(
    string userMessage,
    [EnumeratorCancellation] CancellationToken ct = default)
{
    var chatHistory = new ChatHistory();
    chatHistory.AddUserMessage(userMessage);

    var chatService = _kernel.GetRequiredService<IChatCompletionService>();

    await foreach (var chunk in chatService.GetStreamingChatMessageContentsAsync(
        chatHistory,
        cancellationToken: ct))
    {
        if (!string.IsNullOrEmpty(chunk.Content))
        {
            yield return chunk.Content;
        }
    }
}

In a minimal API endpoint:

app.MapPost("/chat/stream", async (ChatRequest request, ChatService service, HttpResponse response, CancellationToken ct) =>
{
    response.Headers.ContentType = "text/event-stream";
    response.Headers.CacheControl = "no-cache";
    response.Headers.Connection = "keep-alive";

    await foreach (var token in service.StreamResponseAsync(request.Message, ct))
    {
        await response.WriteAsync($"data: {token}\n\n", ct);
        await response.Body.FlushAsync(ct);
    }
});

Tradeoff: streaming disables some middleware that buffers responses. If you sit behind a load balancer or API gateway, verify it passes chunked transfer encoding without buffering. Cloudflare, AWS ALB, and Azure Front Door all support this but may need configuration.

Function calling with native .NET methods

Mistral Large supports function calling. Semantic Kernel maps .NET methods to function schemas via the [KernelFunction] attribute. The framework handles JSON serialization and schema generation.

public sealed class OrderPlugin
{
    private readonly IOrderRepository _orders;

    public OrderPlugin(IOrderRepository orders) => _orders = orders;

    [KernelFunction("get_order_status")]
    [Description("Retrieves the current status of an order by its ID.")]
    public async Task<OrderStatus> GetOrderStatusAsync(
        [Description("The unique order identifier")] string orderId,
        CancellationToken ct = default)
    {
        var order = await _orders.FindAsync(orderId, ct);
        return order?.Status ?? OrderStatus.NotFound;
    }

    [KernelFunction("cancel_order")]
    [Description("Cancels an order if it is still pending.")]
    public async Task<bool> CancelOrderAsync(
        [Description("The unique order identifier")] string orderId,
        CancellationToken ct = default)
    {
        var order = await _orders.FindAsync(orderId, ct);
        if (order?.Status != OrderStatus.Pending) return false;

        order.Status = OrderStatus.Cancelled;
        await _orders.SaveAsync(order, ct);
        return true;
    }
}

Register the plugin with the kernel:

builder.Services.AddKernel()
    .AddMistralLarge(apiKey)
    .Plugins.AddFromType<OrderPlugin>();

Invoke automatically via the chat completion service:

public async Task<string> HandleCustomerQueryAsync(string userMessage, CancellationToken ct = default)
{
    var chatHistory = new ChatHistory();
    chatHistory.AddSystemMessage("You are a customer support agent. Use tools to look up orders.");
    chatHistory.AddUserMessage(userMessage);

    var executionSettings = new MistralAIPromptExecutionSettings
    {
        ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
    };

    var result = await _kernel.GetRequiredService<IChatCompletionService>()
        .GetChatMessageContentAsync(chatHistory, executionSettings, _kernel, ct);

    return result.Content ?? string.Empty;
}

The ToolCallBehavior.AutoInvokeKernelFunctions setting tells SK to execute the matched function and feed the result back to the model in the same turn. Without it, you get a FunctionCallContent in the response that you must handle manually.

Pitfall: Mistral’s function calling expects strict JSON schema compliance. SK generates schemas from your method signatures, but complex types (nested records, dictionaries) can produce schemas the model struggles with. Keep parameters flat — primitives, strings, enums, and simple arrays. If you need complex input, accept a JSON string and deserialize inside the function.

Handling context and token limits

Mistral Large has a 128k token context window. Semantic Kernel doesn’t automatically truncate history. You must manage it.

public sealed class TruncatingChatHistory : ChatHistory
{
    private readonly int _maxTokens;
    private readonly ITokenCounter _tokenCounter;

    public TruncatingChatHistory(int maxTokens, ITokenCounter tokenCounter)
    {
        _maxTokens = maxTokens;
        _tokenCounter = tokenCounter;
    }

    public new void AddMessage(ChatMessageContent message)
    {
        base.AddMessage(message);
        TruncateIfNeeded();
    }

    private void TruncateIfNeeded()
    {
        while (Count > 1 && _tokenCounter.CountTokens(this) > _maxTokens)
        {
            // Remove oldest non-system message
            for (int i = 1; i < Count; i++)
            {
                if (this[i].Role != AuthorRole.System)
                {
                    RemoveAt(i);
                    break;
                }
            }
        }
    }
}

You’ll need a token counter. Mistral uses the mistral-common tokenizer. There’s no official .NET port, but you can approximate with Tiktoken (cl100k_base) or call a tokenization endpoint if Mistral exposes one. For production, consider a gateway that reports usage per request — n4n.ai surfaces per-token metering in response headers, which lets you track consumption without client-side counting.

Structured output with JSON schema

Mistral Large supports constrained JSON output via the response_format parameter. SK exposes this through MistralAIPromptExecutionSettings.

public sealed class SentimentAnalysis
{
    public required string Label { get; init; } // positive, negative, neutral
    public required double Confidence { get; init; }
    public string? Reasoning { get; init; }
}

public async Task<SentimentAnalysis> AnalyzeSentimentAsync(string text, CancellationToken ct = default)
{
    var settings = new MistralAIPromptExecutionSettings
    {
        ResponseFormat = new MistralAIChatResponseFormat
        {
            Type = "json_object",
            JsonSchema = JsonSchema.FromType<SentimentAnalysis>()
        },
        Temperature = 0.1
    };

    var prompt = $$"""
        Analyze the sentiment of the following text. Respond only with valid JSON matching the schema.
        Text: "{{text}}"
        """;

    var result = await _kernel.InvokePromptAsync(prompt, new(settings), ct);
    return JsonSerializer.Deserialize<SentimentAnalysis>(result.GetValue<string>())!;
}

The JsonSchema.FromType<T>() helper generates a schema from your C# type using System.Text.Json serialization attributes. Use [JsonPropertyName] to control field names.

Tradeoff: constrained output increases latency slightly and can fail if the model hallucinates invalid JSON. Set a low temperature and validate the deserialized result. Have a fallback path for parse failures.

Error handling and retries

Mistral’s API returns standard HTTP status codes. The SK connector wraps them in HttpOperationException with the response body. Implement a retry policy for transient failures (429, 5xx).

builder.Services.AddHttpClient<MistralAIClient>(client =>
{
    client.DefaultRequestHeaders.Authorization = new("Bearer", apiKey);
})
.AddStandardResilienceHandler(options =>
{
    options.Retry.MaxRetryAttempts = 3;
    options.Retry.Delay = TimeSpan.FromSeconds(2);
    options.Retry.BackoffType = DelayBackoffType.Exponential;
    options.Retry.UseJitter = true;
    options.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(30);
    options.CircuitBreaker.FailureRatio = 0.5;
    options.CircuitBreaker.MinimumThroughput = 10;
});

The AddStandardResilienceHandler requires Microsoft.Extensions.Http.Resilience (preview as of .NET 8). It handles retries, circuit breaking, and timeouts declaratively.

Pitfall: don’t retry non-idempotent operations like function calls that mutate state. The chat completion endpoint is idempotent for a given conversation history, but if your plugin executes side effects, you need idempotency keys at the application layer.

Testing with a local mock

Unit tests shouldn’t hit the real API. Create a fake IChatCompletionService that returns deterministic responses.

public sealed class FakeChatCompletionService : IChatCompletionService
{
    private readonly IReadOnlyList<string> _responses;
    private int _index;

    public FakeChatCompletionService(params string[] responses) => _responses = responses;

    public IAsyncEnumerable<StreamingChatMessageContent> GetStreamingChatMessageContentsAsync(
        ChatHistory chatHistory,
        PromptExecutionSettings? executionSettings = null,
        Kernel? kernel = null,
        CancellationToken cancellationToken = default)
    {
        var response = _responses[_index % _responses.Length];
        _index++;
        return AsyncEnumerableExtensions.FromSingle(new StreamingChatMessageContent(AuthorRole.Assistant, response));
    }

    public Task<IReadOnlyList<ChatMessageContent>> GetChatMessageContentsAsync(
        ChatHistory chatHistory,
        PromptExecutionSettings? executionSettings = null,
        Kernel? kernel = null,
        CancellationToken cancellationToken = default)
    {
        var response = _responses[_index % _responses.Length];
        _index++;
        return Task.FromResult<IReadOnlyList<ChatMessageContent>>(
            [new ChatMessageContent(AuthorRole.Assistant, response)]);
    }

    public IReadOnlyDictionary<string, object?> Attributes => new Dictionary<string, object?>();
}

Register it in test projects:

services.AddKernel()
    .Services.AddSingleton<IChatCompletionService>(new FakeChatCompletionService("Test response"));

This lets you test prompt construction, function calling flows, and history management without network dependencies.

Deployment considerations

Containerize the application. The Mistral connector uses HttpClient under the hood, so configure HttpClientFactory with appropriate timeouts and connection limits.

FROM mcr.microsoft.com/dotnet/aspnet:8.0
WORKDIR /app
COPY --from=build /app/publish .
ENV ASPNETCORE_URLS=http://+:8080
EXPOSE 8080
ENTRYPOINT ["dotnet", "MyApp.dll"]

Set these environment variables in your deployment:

• MISTRAL_API_KEY — never hardcode this
• MISTRAL_ENDPOINT — optional, defaults to https://api.mistral.ai/v1
• ASPNETCORE_ENVIRONMENT=Production

Monitor the x-ratelimit-remaining and x-ratelimit-reset headers Mistral returns. Build a dashboard around them. If you route through an inference gateway, you get unified rate-limit visibility across providers — n4n.ai forwards provider cache-control hints and rate-limit headers so your observability stack sees them without extra instrumentation.

Common pitfalls summary

When to consider alternatives

Mistral Large via Semantic Kernel works well for .NET teams already invested in the SK ecosystem. Consider alternatives if:

• You need fine-grained control over the request/response pipeline — use the Mistral .NET SDK directly or raw HttpClient
• You’re building a multi-provider system — an inference gateway abstracts provider differences and handles fallback
• You need local/on-premise deployment — Mistral Large isn’t open-weight; look at Mistral 7B, Mixtral, or Llama 3 via Ollama or llama.cpp with SK’s local model connectors

The integration path above gets you to a maintainable, observable, production-ready baseline. Start there, measure, and optimize the specific bottlenecks you encounter.