Wiring up dotnet iasyncenumerable streaming llm responses in C# is the difference between a snappy chat UI and a frozen spinner. Most HTTP client samples block on the full response body; we’ll build a pipeline that yields text deltas as they arrive, respects cancellation, and degrades cleanly when the upstream hiccups.
Step 1: Pick an endpoint and format the request
OpenAI-compatible chat completion streams use Server-Sent Events (SSE) over a plain HTTPS POST with "stream": true. Point the request at any compliant gateway. For example, n4n.ai exposes one OpenAI-compatible endpoint fronting 240+ models with automatic fallback, so the same code works whether the backend is OpenAI, Anthropic, or a local model.
Build the request with System.Net.Http.Json:
using System.Net.Http.Json;
using var client = new HttpClient { BaseAddress = new Uri("https://api.n4n.ai/v1/") };
var requestBody = new
{
model = "gpt-4o-mini",
messages = new[] { new { role = "user", content = "Explain backpressure in one sentence." } },
stream = true
};
using var req = new HttpRequestMessage(HttpMethod.Post, "chat/completions")
{
Content = JsonContent.Create(requestBody)
};
req.Headers.Add("Authorization", "Bearer " + Environment.GetEnvironmentVariable("LLM_API_KEY"));Keep the HttpClient singleton. Creating one per call exhausts sockets under load and adds TLS handshake latency to every token.
Step 2: Open the response stream without buffering
The default HttpClient behavior buffers the entire body before returning. For token streaming you must pass HttpCompletionOption.ResponseHeadersRead and read the stream incrementally. This is the foundation of any dotnet iasyncenumerable streaming llm client.
using var resp = await client.SendAsync(req, HttpCompletionOption.ResponseHeadersRead, ct);
resp.EnsureSuccessStatusCode();
var stream = await resp.Content.ReadAsStreamAsync(ct);
using var reader = new StreamReader(stream);At this point no tokens have been parsed. The connection stays open until the model finishes or the client disconnects. If you omit ResponseHeadersRead, the await returns only after the last token, defeating the purpose.
Step 3: Parse SSE lines into a token sequence
SSE frames look like data: {json}\n\n. The stream ends with data: [DONE]. Each JSON payload contains choices[0].delta.content for incremental text. Comments starting with : are heartbeats and must be ignored.
Write an iterator that reads lines, skips non-data lines, and yields the content string:
using System.Text.Json;
async IAsyncEnumerable<string> ReadTokensAsync(StreamReader reader,
[EnumeratorCancellation] CancellationToken ct = default)
{
while (!reader.EndOfStream)
{
var line = await reader.ReadLineAsync(ct);
if (string.IsNullOrWhiteSpace(line) || !line.StartsWith("data:")) continue;
var payload = line["data:".Length..].Trim();
if (payload == "[DONE]") yield break;
try
{
using var doc = JsonDocument.Parse(payload);
if (doc.RootElement.TryGetProperty("choices", out var choices) &&
choices[0].TryGetProperty("delta", out var delta) &&
delta.TryGetProperty("content", out var content))
{
yield return content.GetString() ?? string.Empty;
}
}
catch (JsonException)
{
// Skip malformed or partial frames; upstreams occasionally send ping comments.
}
}
}The [EnumeratorCancellation] attribute wires the CancellationToken into the state machine so a disconnected client stops the loop immediately. Never call ReadLineAsync without await—synchronous reads block the thread pool under slow networks.
Step 4: Package it in a reusable client
A small wrapper keeps call sites clean and centralizes error handling. This version throws on non-success status before streaming and surfaces upstream errors as InvalidOperationException.
public sealed class LLMStreamClient
{
private readonly HttpClient _client;
public LLMStreamClient(HttpClient client) => _client = client;
public async IAsyncEnumerable<string> StreamChatAsync(
string model,
string prompt,
[EnumeratorCancellation] CancellationToken ct = default)
{
var body = new
{
model,
messages = new[] { new { role = "user", content = prompt } },
stream = true
};
using var req = new HttpRequestMessage(HttpMethod.Post, "chat/completions")
{
Content = JsonContent.Create(body)
};
req.Headers.Add("Authorization", "Bearer " + Environment.GetEnvironmentVariable("LLM_API_KEY"));
using var resp = await _client.SendAsync(req, HttpCompletionOption.ResponseHeadersRead, ct);
if (!resp.IsSuccessStatusCode)
throw new InvalidOperationException($"Upstream returned {(int)resp.StatusCode}");
var stream = await resp.Content.ReadAsStreamAsync(ct);
using var reader = new StreamReader(stream);
await foreach (var token in ReadTokensAsync(reader, ct))
yield return token;
}
private static async IAsyncEnumerable<string> ReadTokensAsync(
StreamReader reader, [EnumeratorCancellation] CancellationToken ct = default)
{
while (!reader.EndOfStream)
{
var line = await reader.ReadLineAsync(ct);
if (string.IsNullOrWhiteSpace(line) || !line.StartsWith("data:")) continue;
var payload = line["data:".Length..].Trim();
if (payload == "[DONE]") yield break;
try
{
using var doc = JsonDocument.Parse(payload);
if (doc.RootElement.TryGetProperty("choices", out var choices) &&
choices[0].TryGetProperty("delta", out var delta) &&
delta.TryGetProperty("content", out var content))
{
yield return content.GetString() ?? string.Empty;
}
}
catch (JsonException) { }
}
}
}If you route through a gateway that honors client routing directives and forwards provider cache-control hints, those headers pass through transparently; no code change needed.
Step 5: Consume the stream in a real app
In a console tool, await foreach prints tokens as they land:
var client = new LLMStreamClient(new HttpClient { BaseAddress = new Uri("https://api.n4n.ai/v1/") });
await foreach (var token in client.StreamChatAsync("gpt-4o-mini", "Write a haiku about TCP.", default))
{
Console.Write(token);
await Task.Yield();
}For ASP.NET Core, return the sequence from a minimal API and let the framework serialize it as NDJSON, or write SSE manually for browser EventSource compatibility:
app.MapGet("/stream", async (HttpContext http, LLMStreamClient client, CancellationToken ct) =>
{
http.Response.ContentType = "text/event-stream";
await foreach (var token in client.StreamChatAsync("gpt-4o-mini", "Stream me a story.", ct))
{
await http.Response.WriteAsync($"data: {token}\n\n", ct);
await http.Response.Body.FlushAsync(ct);
}
});The FlushAsync call pushes each chunk to the browser immediately. Omit it and you’ll batch unpredictably.
Blazor integration
In a Blazor Server component, inject the client and render incrementally:
@inject LLMStreamClient Client
<pre>@output</pre>
@code {
private string output = "";
protected override async Task OnInitializedAsync()
{
await foreach (var token in Client.StreamChatAsync("gpt-4o-mini", "Explain Razor rendering.", default))
{
output += token;
StateHasChanged();
}
}
}StateHasChanged per token is fine for low-volume streams; for high throughput, throttle UI updates with a Timer or batch every 50 ms.
Step 6: Verify success and handle edge cases
Run the console app with a valid key. You should see text appear word-by-word, not all at once after a delay. If you get a 401, the header is malformed. If the process hangs, you forgot ResponseHeadersRead.
Test the HTTP endpoint with curl:
curl -N http://localhost:5000/streamThe -N flag disables curl’s buffering. You should receive data: ... frames incrementally.
Wire a 30-second CancellationTokenSource in tests to confirm the stream aborts cleanly. In production, hook the token to HttpContext.RequestAborted so closing the tab terminates the upstream call.
Watch for partial JSON. Upstreams occasionally emit a heartbeat comment (": ping") or split a frame across TCP packets; StreamReader.ReadLineAsync handles the split, but never assume every data: line parses. Wrap JsonDocument.Parse in try/catch and skip malformed lines.
If you use a gateway with per-token usage metering, the final frame includes a usage object. Capture it after the loop by inspecting the last parsed payload, or add a UsageReceived callback in your client. That metric is the only reliable way to bill or rate-limit multi-tenant calls.
Gotchas that bite in production
- Synchronous reads: Calling
reader.ReadLine()withoutawaitblocks the thread pool under slow networks. Always await. - Disposing HttpClient: Don’t wrap the shared client in
usingat the call site. Let DI own it. - Encoding: SSE is UTF-8.
StreamReaderdefaults to UTF-8 with BOM detection off; that’s fine. - Backpressure: If the consumer is slower than the producer (e.g., writing to a sluggish WebSocket), the
await foreachloop naturally pauses reading, which applies backpressure to the TCP stack. Don’t artificially buffer tokens in aList. - Timeouts: Set
HttpClient.TimeouttoInfiniteTimeSpanwhen streaming long completions; otherwise the client cancels mid-stream.
The pattern above gives you a cancellable, backpressure-aware dotnet iasyncenumerable streaming llm pipeline with about 100 lines of code. It compiles on .NET 8, runs in Lambda or a container, and degrades predictably when the model backend goes 503.