Streaming chat completions in Laravel with SSE

Wiring up laravel sse streaming chat completions against a remote LLM endpoint is straightforward once you stop treating the response like a normal JSON call. This guide walks through a production-shaped implementation using Laravel’s streaming responses and an OpenAI-compatible chat completions API, so tokens reach the browser as they are generated.

Step 1: Define the route and controller

Laravel’s response()->stream() is the right primitive. It keeps the PHP process alive and pushes chunks to the client without buffering the full payload.

Add a POST route that accepts the user message and returns an event stream:

// routes/web.php
Route::post('/chat/stream', [App\Http\Controllers\ChatController::class, 'stream']);

The controller method will open a streaming HTTP request to the LLM gateway and pipe SSE frames back to the browser. Keep the controller thin; put the HTTP client logic in a dedicated service if your app grows.

Step 2: Open a streaming request to the LLM endpoint

Use Guzzle with the stream option. The chat completions endpoint must receive "stream": true in the JSON body. Point the base URL at any OpenAI-compatible gateway. For example, n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and automatically fails over when a provider is rate-limited, which removes a class of retry code from your Laravel app.

// app/Services/LlmStreamer.php
namespace App\Services;

use GuzzleHttp\Client;

class LlmStreamer
{
    public function stream(array $messages, string $apiKey, string $baseUrl)
    {
        $client = new Client(['base_uri' => $baseUrl]);
        $response = $client->post('/v1/chat/completions', [
            'stream' => true,
            'headers' => [
                'Authorization' => 'Bearer ' . $apiKey,
                'Content-Type' => 'application/json',
            ],
            'json' => [
                'model' => 'gpt-4o-mini',
                'messages' => $messages,
                'stream' => true,
            ],
        ]);

        return $response->getBody();
    }
}

The returned body is a GuzzleHttp\Psr7\Stream that yields raw bytes. Do not call getContents() — that blocks until the stream closes.

Step 3: Pipe SSE frames to the browser

The LLM endpoint emits Server-Sent Events: lines prefixed with data: , separated by blank lines, ending with data: [DONE]. Your Laravel closure must read those lines and echo them with the correct SSE content type.

// app/Http/Controllers/ChatController.php
namespace App\Http\Controllers;

use App\Services\LlmStreamer;
use Illuminate\Http\Request;

class ChatController extends Controller
{
    public function stream(Request $request)
    {
        $messages = $request->input('messages', []);
        $streamer = new LlmStreamer();
        $body = $streamer->stream($messages, config('services.llm.key'), config('services.llm.base'));

        return response()->stream(function () use ($body) {
            $buffer = '';
            while (!$body->eof()) {
                $chunk = $body->read(1024);
                $buffer .= $chunk;
                // SSE frames end with double newline
                while (($pos = strpos($buffer, "\n\n")) !== false) {
                    $frame = substr($buffer, 0, $pos);
                    $buffer = substr($buffer, $pos + 2);
                    echo $frame . "\n\n";
                    ob_flush();
                    flush();
                }
            }
            if ($buffer !== '') {
                echo $buffer . "\n\n";
                ob_flush();
                flush();
            }
        }, 200, [
            'Content-Type' => 'text/event-stream',
            'Cache-Control' => 'no-cache',
            'X-Accel-Buffering' => 'no',
        ]);
    }
}

The X-Accel-Buffering: no header disables nginx proxy buffering so frames are not held upstream. If you deploy behind Apache, ensure mod_deflate is off for this path.

Step 4: Parse and forward only the delta content

Raw frames from the LLM look like:

data: {"id":"chatcmpl-123","choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}

You can forward them verbatim, but it is cleaner to reshape the payload so the frontend gets only the text delta. Modify the inner loop:

while (($pos = strpos($buffer, "\n\n")) !== false) {
    $frame = substr($buffer, 0, $pos);
    $buffer = substr($buffer, $pos + 2);
    if (str_starts_with($frame, 'data: ')) {
        $payload = substr($frame, 6);
        if ($payload === '[DONE]') {
            echo "event: done\ndata: {}\n\n";
            continue;
        }
        $json = json_decode($payload, true);
        $delta = $json['choices'][0]['delta']['content'] ?? '';
        if ($delta !== '') {
            echo "data: " . json_encode(['token' => $delta]) . "\n\n";
            ob_flush();
            flush();
        }
    }
}

This narrows the contract between Laravel and the browser. The laravel sse streaming chat completions pipeline now emits data: {"token":"..."} lines plus a custom done event.

Step 5: Capture usage and honor cache hints

The final frame from an OpenAI-compatible endpoint includes a usage object with prompt and completion tokens. Even when streaming, capture it for per-token metering:

$usage = null;
// inside the frame loop, after json_decode:
if (isset($json['usage'])) {
    $usage = $json['usage'];
}
// after the loop:
if ($usage) {
    \Log::info('llm_usage', $usage);
}

If your gateway forwards provider cache-control hints (some send x-cache: HIT or similar headers on the HTTP response), read them from $response->getHeaderLine('x-cache') before entering the stream loop and log accordingly. n4n.ai honors client routing directives and forwards provider cache-control hints, so you can attribute cost savings without extra plumbing.

Step 6: Consume the stream in the browser

EventSource only supports GET, but our route is POST. Use fetch with a readable stream reader instead:

async function streamChat(messages: any[]) {
  const res = await fetch('/chat/stream', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
  });
  const reader = res.body!.getReader();
  const decoder = new TextDecoder();
  let buffer = '';
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });
    const parts = buffer.split('\n\n');
    buffer = parts.pop() ?? '';
    for (const part of parts) {
      if (part.startsWith('data: ')) {
        const data = JSON.parse(part.slice(6));
        if (data.token) process.stdout.write(data.token);
      }
    }
  }
}

This TypeScript snippet writes tokens to the console; in a real UI you would append to a DOM node.

Step 7: Verify the end-to-end flow

Start the Laravel dev server and run a curl command that mimics the frontend POST:

curl -N -X POST http://localhost:8000/chat/stream \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Say hi in 5 words"}]}'

You should see data: {...} lines appear incrementally, followed by the done event. If curl prints the whole response at once, check for proxy buffering or missing X-Accel-Buffering header. In the browser, open the Network tab, select the request, and watch the Response section fill token by token.

Step 8: Production hardening

• Set a reasonable request_timeout on the Guzzle client (e.g., 60 seconds) and catch GuzzleHttp\Exception\ConnectException to emit an SSE error frame.
• Run PHP with FPM and ensure output_buffering = Off in php.ini for the stream path.
• If you use Laravel Octane, avoid long-lived streams that block workers; prefer a dedicated queue or a stateless gateway call.
• For laravel sse streaming chat completions at scale, place a CDN or load balancer that supports chunked transfer encoding without aggregation.

The pattern above keeps your Laravel app as a thin, resilient proxy: it opens one upstream stream, parses frames, and forwards them with minimal latency. That is all you need to ship a responsive chat UI on top of any OpenAI-compatible model backend.