First Semantic Kernel app with n4n.ai and Llama 3.3 70B

Semantic Kernel has become the go-to orchestration layer for .NET and Python developers who want to compose LLMs with plugins, planners, and memory without rewriting their application logic every time a new model drops. This semantic kernel n4n.ai llama 3.3 70b tutorial walks you through a working console app that streams chat, calls a local function, and falls back automatically if the primary provider degrades — all in about 30 minutes.

Prerequisites

You need three things installed before starting:

• .NET 8 SDK (or .NET 9 preview if you prefer)
• An n4n.ai API key — sign up at n4n.ai and copy the key from the dashboard
• A code editor — VS Code with the C# Dev Kit or Visual Studio 2022

Verify the SDK:

dotnet --version
# 8.0.x or 9.0.x

Create the project

mkdir SkLlamaDemo && cd SkLlamaDemo
dotnet new console --framework net8.0
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.SemanticKernel.Connectors.OpenAI

The OpenAI connector works because n4n.ai exposes an OpenAI-compatible endpoint. No custom provider code required.

Configure the kernel

Replace Program.cs with the following. The kernel points at the n4n.ai base address, authenticates with your key, and requests meta-llama/llama-3.3-70b-instruct — the model identifier n4n.ai uses for Llama 3.3 70B.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.ChatCompletion;

var apiKey = Environment.GetEnvironmentVariable("N4N_API_KEY")
    ?? throw new InvalidOperationException("Set N4N_API_KEY env var");

var builder = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(
        modelId: "meta-llama/llama-3.3-70b-instruct",
        apiKey: apiKey,
        endpoint: new Uri("https://api.n4n.ai/v1"),
        serviceId: "llama33-70b"
    );

var kernel = builder.Build();

Set the environment variable in your shell before running:

export N4N_API_KEY="sk-..."   # Linux/macOS
# $env:N4N_API_KEY="sk-..."   # PowerShell

Run a quick sanity check:

dotnet run

No output yet — that’s expected. The kernel constructs without throwing.

Stream a chat completion

Semantic Kernel’s IChatCompletionService supports streaming out of the box. Add this after the kernel build:

var chat = kernel.GetRequiredService<IChatCompletionService>();

var history = new ChatHistory("You are a concise technical assistant.");

Console.Write("User > ");
var userInput = Console.ReadLine();

history.AddUserMessage(userInput!);

await foreach (var chunk in chat.GetStreamingChatMessageContentsAsync(
    history,
    executionSettings: new OpenAIPromptExecutionSettings { MaxTokens = 512 },
    kernel: kernel))
{
    Console.Write(chunk);
}
Console.WriteLine();

Run it:

dotnet run

Expected interaction:

User > Explain the difference between a struct and a class in C# in two sentences.
A struct is a value type allocated on the stack (or inline in an array), while a class is a reference type allocated on the heap. Structs copy by value; classes copy by reference.

The stream prints token-by-token with no buffering code on your side.

Add a native function (plugin)

Semantic Kernel shines when you attach deterministic code as callable functions. Create a TimePlugin.cs next to Program.cs:

using Microsoft.SemanticKernel;

public class TimePlugin
{
    [KernelFunction("get_utc_now")]
    [Description("Returns the current UTC time in ISO 8601 format.")]
    public string GetUtcNow() => DateTime.UtcNow.ToString("o");
}

Register it and enable auto-invocation:

// in Program.cs, after kernel.Build()
kernel.Plugins.AddFromType<TimePlugin>("Time");

var executionSettings = new OpenAIPromptExecutionSettings
{
    MaxTokens = 512,
    ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};

Update the streaming loop to pass executionSettings:

await foreach (var chunk in chat.GetStreamingChatMessageContentsAsync(
    history,
    executionSettings: executionSettings,
    kernel: kernel))
{
    Console.Write(chunk);
}

Run again and ask:

User > What time is it right now in UTC?

Expected output (timestamp will differ):

User > What time is it right now in UTC?
2025-01-15T14:32:07.1234567Z

The model emitted a tool call, the kernel executed TimePlugin.GetUtcNow, and the result streamed back as part of the same response — no manual orchestration.

Add a semantic function (prompt template)

Native functions are C#; semantic functions are prompt templates stored as files. Create a Plugins/Summarize/skprompt.txt:

Summarize the following text in exactly one sentence. Preserve key numbers and names.

{{$input}}

And a config.json beside it:

{
  "schema": 1,
  "type": "completion",
  "description": "One-sentence summarizer",
  "input_variables": [
    { "name": "input", "description": "Text to summarize", "required": true }
  ],
  "execution_settings": {
    "default": { "max_tokens": 128, "temperature": 0.2 }
  }
}

Register the plugin directory:

kernel.Plugins.AddFromPromptDirectory("Plugins");

Now invoke it explicitly from code (auto-invoke works too, but explicit calls are clearer for demos):

var summarize = kernel.Plugins["Summarize"]["Summarize"];

var longText = """
    The Mars Sample Return campaign, a joint NASA-ESA effort, plans to launch
    the Sample Retrieval Lander in 2028 carrying a Mars Ascent Vehicle.
    Perseverance has already cached 24 rock cores in Three Forks depot.
    """;

var result = await kernel.InvokeAsync(summarize, new() { ["input"] = longText });
Console.WriteLine($"\nSummary: {result}");

Run:

Summary: NASA and ESA plan to launch a 2028 lander to retrieve 24 Mars rock cores cached by Perseverance.

Handle fallback automatically

One reason teams put n4n.ai in front of multiple providers is automatic fallback when a model is rate-limited or degraded. The gateway honors the x-n4n-fallback header and returns provider cache-control hints, but you don’t need special client code — just catch the standard HttpRequestException and retry once with a short backoff. Semantic Kernel’s OpenAIPromptExecutionSettings lets you set MaxRetries on the underlying HttpClient via a custom HttpClientFactory, but the simplest production pattern is a wrapper:

static async Task<T> WithFallback<T>(Func<Task<T>> action, int maxAttempts = 2)
{
    for (int attempt = 1; ; attempt++)
    {
        try { return await action(); }
        catch (HttpRequestException ex) when (attempt < maxAttempts)
        {
            await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, attempt)));
        }
    }
}

Wrap the streaming call:

await WithFallback(async () =>
{
    await foreach (var chunk in chat.GetStreamingChatMessageContentsAsync(
        history, executionSettings, kernel))
    {
        Console.Write(chunk);
    }
    Console.WriteLine();
    return true;
});

If the primary provider returns 429 or 5xx, the gateway routes to the next healthy provider and your app sees a successful response on the retry — no code change, no new deployment.

Persist chat history to disk (optional)

For a real app you’ll want history surviving restarts. Semantic Kernel doesn’t prescribe storage, but a JSON file is five lines:

var historyPath = "chat_history.json";

if (File.Exists(historyPath))
{
    var json = File.ReadAllText(historyPath);
    history = JsonSerializer.Deserialize<ChatHistory>(json, new JsonSerializerOptions
    {
        PropertyNameCaseInsensitive = true
    })!;
}

// ... after each turn ...
File.WriteAllText(historyPath, JsonSerializer.Serialize(history));

Add using System.Text.Json; at the top. The history object round-trips cleanly because ChatHistory is a plain list of ChatMessageContent.

Run the complete flow

Final Program.cs in one piece:

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.ChatCompletion;
using System.Text.Json;

var apiKey = Environment.GetEnvironmentVariable("N4N_API_KEY")
    ?? throw new InvalidOperationException("Set N4N_API_KEY env var");

var builder = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(
        modelId: "meta-llama/llama-3.3-70b-instruct",
        apiKey: apiKey,
        endpoint: new Uri("https://api.n4n.ai/v1"),
        serviceId: "llama33-70b"
    );

var kernel = builder.Build();
kernel.Plugins.AddFromType<TimePlugin>("Time");
kernel.Plugins.AddFromPromptDirectory("Plugins");

var chat = kernel.GetRequiredService<IChatCompletionService>();
var history = new ChatHistory("You are a concise technical assistant.");

var historyPath = "chat_history.json";
if (File.Exists(historyPath))
{
    var json = File.ReadAllText(historyPath);
    history = JsonSerializer.Deserialize<ChatHistory>(json, new JsonSerializerOptions
    {
        PropertyNameCaseInsensitive = true
    })!;
}

var executionSettings = new OpenAIPromptExecutionSettings
{
    MaxTokens = 512,
    ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};

Console.WriteLine("Chat started. Type 'exit' to quit.\n");

while (true)
{
    Console.Write("User > ");
    var input = Console.ReadLine();
    if (string.Equals(input, "exit", StringComparison.OrdinalIgnoreCase)) break;

    history.AddUserMessage(input!);

    await WithFallback(async () =>
    {
        await foreach (var chunk in chat.GetStreamingChatMessageContentsAsync(
            history, executionSettings, kernel))
        {
            Console.Write(chunk);
        }
        Console.WriteLine();
        return true;
    });

    File.WriteAllText(historyPath, JsonSerializer.Serialize(history));
}

static async Task<T> WithFallback<T>(Func<Task<T>> action, int maxAttempts = 2)
{
    for (int attempt = 1; ; attempt++)
    {
        try { return await action(); }
        catch (HttpRequestException ex) when (attempt < maxAttempts)
        {
            await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, attempt)));
        }
    }
}

Run it:

dotnet run

Sample session:

Chat started. Type 'exit' to quit.

User > Summarize the Mars Sample Return plan in one sentence.
Summary: NASA and ESA plan to launch a 2028 lander to retrieve 24 Mars rock cores cached by Perseverance.

User > What time is it in UTC?
2025-01-15T14:45:12.3456789Z

User > exit

The history file now contains four messages (system, user, assistant, user, assistant, user, assistant) and survives a process restart.

What you’ve built

• A kernel pointed at an OpenAI-compatible endpoint serving Llama 3.3 70B
• Streaming chat with token-level output
• A native plugin (TimePlugin) auto-invoked via tool calling
• A semantic plugin (Summarize) loaded from prompt files
• Automatic fallback via a tiny retry wrapper — the gateway handles provider switching
• Persisted history so conversations survive restarts

All of this runs on the model identifier meta-llama/llama-3.3-70b-instruct behind a single base URL. Swap the model ID to anthropic/claude-3.5-sonnet or openai/gpt-4o and the same code works — no SDK changes, no new packages.

Next steps

• Add a vector memory connector (Qdrant, Pinecone, or the in-memory VolatileMemoryStore) for RAG
• Implement a planner (FunctionCallingStepwisePlanner) for multi-step tasks
• Containerize the console app for CI/CD — the only runtime dependency is the .NET runtime and the N4N_API_KEY secret
• Monitor per-token usage via the gateway’s usage endpoint if you need cost attribution per tenant

The semantic kernel n4n.ai llama 3.3 70b tutorial pattern scales from prototype to production because the orchestration layer stays decoupled from the model provider. You write plugins once; the gateway handles availability, latency routing, and usage metering.