Semantic Kernel enterprise tutorial: n4n.ai cost tracking

Enterprises adopting Semantic Kernel face a blunt problem: without granular token metering, LLM spend becomes a black box. For semantic kernel enterprise cost tracking n4n.ai provides a single OpenAI-compatible endpoint with per-token usage metering across 240+ models, which we can tap directly from .NET. This guide lays out an ordered path to capture, attribute, and persist those costs without bolting on a separate billing system.

1. Point Semantic Kernel at the gateway

Semantic Kernel’s OpenAI connector accepts a custom endpoint URI. That is all you need to route through an OpenRouter-class proxy instead of api.openai.com.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;

var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(
    modelId: "anthropic/claude-3.5-sonnet",
    apiKey: Environment.GetEnvironmentVariable("N4N_API_KEY")!,
    endpoint: new Uri("https://api.n4n.ai/v1")
);
var kernel = builder.Build();

The modelId string is passed through verbatim. The gateway resolves it to an underlying provider, so you can swap models in config without code changes. Keep the key in a secret store, not in source.

2. Extract usage from response metadata

The OpenAI connector stashes token counts in ChatMessageContent.Metadata. The exact key is "Usage" in current SK builds, but verify against your package version—the shape has shifted between previews.

var chat = kernel.GetRequiredService<IChatCompletionService>();
var history = new ChatHistory("You are a concise analyst.");
history.AddUserMessage("Summarize the Q3 board deck.");

var reply = await chat.GetChatMessageContentAsync(history, kernel: kernel);

if (reply.Metadata?.TryGetValue("Usage", out var usageRaw) == true)
{
    var usage = (OpenAIUsage)usageRaw;
    var record = new CostRecord
    {
        Model = reply.Metadata.TryGetValue("Model", out var m) ? (string)m : "unknown",
        PromptTokens = usage.PromptTokens,
        CompletionTokens = usage.CompletionTokens,
        Timestamp = DateTime.UtcNow
    };
}

Do not assume the requested model equals the billed model. The metadata Model field reflects what actually served the token.

3. Wrap the service with a cost-tracking decorator

Scattering extraction logic across call sites breaks fast. Use a decorator around IChatCompletionService so every completion funnels through one recorder.

public class MeteredChatService : IChatCompletionService
{
    private readonly IChatCompletionService _inner;
    private readonly ICostSink _sink;

    public MeteredChatService(IChatCompletionService inner, ICostSink sink)
        => (_inner, _sink) = (inner, sink);

    public async Task<IReadOnlyList<ChatMessageContent>> GetChatMessageContentsAsync(
        ChatHistory chatHistory,
        PromptExecutionSettings? executionSettings = null,
        Kernel? kernel = null,
        CancellationToken cancellationToken = default)
    {
        var result = await _inner.GetChatMessageContentsAsync(
            chatHistory, executionSettings, kernel, cancellationToken);

        foreach (var msg in result)
        {
            if (msg.Metadata?.TryGetValue("Usage", out var u) == true)
            {
                var usage = (OpenAIUsage)u;
                await _sink.RecordAsync(new CostRecord
                {
                    Model = msg.Metadata.TryGetValue("Model", out var m) ? (string)m : "unknown",
                    PromptTokens = usage.PromptTokens,
                    CompletionTokens = usage.CompletionTokens,
                    Timestamp = DateTime.UtcNow
                });
            }
        }
        return result;
    }

    // Remaining interface members delegate to _inner
}

Register it after the built-in service is configured. In practice, build the kernel, pull the existing IChatCompletionService, and re-add your decorator:

var inner = kernel.GetRequiredService<IChatCompletionService>();
kernel.Services.AddSingleton<IChatCompletionService>(
    new MeteredChatService(inner, new OtelCostSink()));

If you use the kernel factory pattern, wrap inside KernelBuilder.Services with a factory lambda that resolves the inner type from the container.

4. Persist and attribute spend

A cost record with only token counts is useless for chargeback. Extend the record with correlation dimensions your finance team actually queries.

public record CostRecord(
    string Model,
    int PromptTokens,
    int CompletionTokens,
    string? TenantId,
    string? ConversationId,
    DateTime Timestamp);

Push to OpenTelemetry metrics for real-time dashboards, or buffer into a relational table for monthly invoices. A channel-backed sink avoids blocking the LLM call:

public class BufferedCostSink : ICostSink
{
    private readonly Channel<CostRecord> _channel = Channel.CreateUnbounded<CostRecord>();
    private readonly Task _consumer;

    public BufferedCostSink(IDbConnection db) =>
        _consumer = Task.Run(async () =>
        {
            await foreach (var r in _channel.Reader.ReadAllAsync())
                await db.ExecuteAsync("INSERT INTO cost ...", r);
        });

    public ValueTask RecordAsync(CostRecord r) => _channel.Writer.WriteAsync(r);
}

Tradeoff: buffered writes lag by milliseconds to seconds. If you need synchronous audit trails, accept the latency tax or use a transactional outbox.

5. Account for fallback and caching

The gateway performs automatic fallback when a provider is rate-limited or degraded. Your requested modelId might be openai/gpt-4o, but the response metadata could show anthropic/claude-3.5-sonnet if OpenAI was unavailable. Always bill on returned metadata, not the request string.

n4n.ai forwards provider cache-control hints. Some providers report cached prompt tokens separately. The raw JSON may look like:

{
  "usage": {
    "prompt_tokens": 1200,
    "completion_tokens": 300,
    "total_tokens": 1500,
    "cached_tokens": 1000
  }
}

If your internal chargeback differentiates cached vs. fresh tokens, parse the extended fields. SK’s OpenAIUsage may not surface cached_tokens; drop to the raw Metadata["Usage"] dictionary and deserialize manually.

6. Common pitfalls and tradeoffs

Double counting in multi-turn chat. If you resend the full ChatHistory every turn, prompt tokens grow linearly with conversation length. Track cumulative per ConversationId and alert when a single session crosses a threshold.

Streaming blind spots. With StreamingChatMessageContent, usage often appears only in the final chunk’s metadata. If you record on the first message, you will log zero tokens. Collect the whole stream before extracting.

Synchronous sinks on the hot path. A naive await db.SaveChangesAsync() inside the decorator adds 5–15 ms per call. In high-throughput services, that compounds. Use the channel pattern from section 4.

Model alias drift. A model string like mistral/mixtral-8x7b can map to different underlying snapshots. Pin versions in config and reconcile with the gateway’s usage export monthly.

Ignoring function calls. Semantic Kernel plugins invoke text or chat completions under the hood. If you use kernel.InvokeAsync with non-chat functions, wrap ITextCompletionService too, or you will undercount.

7. Minimal verification script

Before wiring this into production, run a probe to confirm metadata flows end to end.

export N4N_API_KEY=sk-...
dotnet run --project CostProbe.csproj

CostProbe should print the Metadata dictionary of a single completion. If Usage is absent, check SK version or that the endpoint returns OpenAI-compatible usage (some minimal proxies strip it).

8. Where to go next

Once per-token records land in your metrics store, build a simple Grafana panel grouped by Model and TenantId. Set hard caps via the gateway’s routing directives if you need to throttle noisy tenants. The decorator pattern here extends cleanly to embeddings and image endpoints—same metadata extraction, different record shape. Semantic Kernel gives you the hook; the rest is disciplined accounting.