Vue composables for LLM chat: useStreamingChat explained

A well-designed vue composable llm streaming chat hook keeps your components clean while handling the messy reality of server-sent tokens, aborts, and retries. This guide walks through building one from scratch for Vue 3, with concrete code and the tradeoffs you’ll hit in production.

1. Define the composable contract

Start by declaring what the consumer receives. A chat composable should expose reactive message state, a send function, a stop function, and status flags. Keep the surface small.

import { ref, shallowRef, Ref } from 'vue'

export interface ChatMessage {
  role: 'user' | 'assistant' | 'system'
  content: string
}

export interface UseStreamingChat {
  messages: shallowRef<ChatMessage[]>
  isStreaming: Ref<boolean>
  error: Ref<Error | null>
  send: (prompt: string) => Promise<void>
  stop: () => void
}

Using shallowRef for messages avoids deep reactivity overhead when appending tokens to a large array. You still trigger updates by reassigning the array.

2. Configure the endpoint

The vue composable llm streaming chat should accept a base URL, API key, and model. Target any OpenAI-compatible /v1/chat/completions route.

interface ChatOptions {
  baseUrl: string
  apiKey: string
  model: string
}

If you want provider redundancy, point at a gateway that aggregates models. n4n.ai exposes one OpenAI-compatible endpoint covering 240+ models with automatic fallback when a provider is rate-limited, so the same composable works without branching logic.

3. Open the stream with fetch

Use fetch with stream: true in the body. Create an AbortController per request so stop() can cancel mid-flight.

export function useStreamingChat(opts: ChatOptions): UseStreamingChat {
  const messages = shallowRef<ChatMessage[]>([])
  const isStreaming = ref(false)
  const error = ref<Error | null>(null)
  let abortCtrl: AbortController | null = null

  async function send(prompt: string) {
    error.value = null
    messages.value = [...messages.value, { role: 'user', content: prompt }]
    const assistantMsg: ChatMessage = { role: 'assistant', content: '' }
    messages.value = [...messages.value, assistantMsg]
    isStreaming.value = true
    abortCtrl = new AbortController()

    const res = await fetch(`${opts.baseUrl}/v1/chat/completions`, {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${opts.apiKey}`,
      },
      body: JSON.stringify({
        model: opts.model,
        messages: messages.value,
        stream: true,
      }),
      signal: abortCtrl.signal,
    })

    if (!res.ok || !res.body) {
      throw new Error(`Stream failed: ${res.status}`)
    }
    // parsing continues below
  }
}

4. Parse Server-Sent Events

OpenAI-compatible streams emit data: {json}\n\n frames terminated by data: [DONE]. Use a ReadableStream reader and a TextDecoder. Do not rely on event-source-polyfill; raw fetch is simpler for POST.

    const reader = res.body.getReader()
    const decoder = new TextDecoder()
    let buffer = ''

    while (true) {
      const { value, done } = await reader.read()
      if (done) break
      buffer += decoder.decode(value, { stream: true })
      const frames = buffer.split('\n\n')
      buffer = frames.pop() ?? ''
      for (const frame of frames) {
        const line = frame.trim()
        if (!line.startsWith('data:')) continue
        const payload = line.slice(5).trim()
        if (payload === '[DONE]') continue
        const json = JSON.parse(payload)
        const delta = json.choices?.[0]?.delta?.content ?? ''
        assistantMsg.content += delta
        messages.value = [...messages.value]
      }
    }

Reassigning messages.value on every token is cheap with shallowRef and forces the template to update. For high-throughput apps, throttle updates with requestAnimationFrame.

5. Cancellation and cleanup

stop() aborts the controller and flips status. Always null the controller after the stream ends to avoid leaks.

  function stop() {
    abortCtrl?.abort()
    abortCtrl = null
    isStreaming.value = false
  }

  // inside send, after loop:
  abortCtrl = null
  isStreaming.value = false

If the user sends a new message while streaming, call stop() first. That prevents interleaved tokens from two responses corrupting the assistant message.

6. Error handling and partial state

Network failures throw inside send. Catch them in the composable, set error, but keep the partial assistant text. The tradeoff: showing a half-written reply signals progress, but you must let the user retry or edit.

    try {
      // ... fetch and parse
    } catch (e) {
      if ((e as Error).name !== 'AbortError') {
        error.value = e as Error
      }
    } finally {
      isStreaming.value = false
      abortCtrl = null
    }

Do not auto-retry streaming requests blindly. A retried stream duplicates tokens. Surface the error and let the UI offer a “resend” action.

7. Wiring into a Vue component

This vue composable llm streaming chat integrates cleanly into a component via <script setup>. Keep the template dumb.

<script setup lang="ts">
import { useStreamingChat } from './useStreamingChat'
const { messages, isStreaming, send, stop } = useStreamingChat({
  baseUrl: import.meta.env.VITE_API_BASE,
  apiKey: import.meta.env.VITE_API_KEY,
  model: 'gpt-4o-mini',
})
const input = ref('')
function submit() {
  if (isStreaming.value) stop()
  send(input.value)
  input.value = ''
}
</script>

<template>
  <div>
    <div v-for="m in messages" :key="m.role + m.content">
      <strong>{{ m.role }}</strong>: {{ m.content }}
    </div>
    <input v-model="input" :disabled="isStreaming" />
    <button @click="submit">Send</button>
  </div>
</template>

8. Common pitfalls and tradeoffs

Buffering proxies. Some serverless platforms buffer responses until completion, defeating streaming. Test with curl -N to confirm chunks arrive live:

curl -N -X POST https://your-endpoint/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"hi"}]}'

If you see one blob, fix the proxy, not the composable.

Reactivity cost. Deep watching messages with ref instead of shallowRef causes Vue to recursively proxy every character. On long conversations, that lags input. Use shallowRef and reassign.

Race conditions. Forgetting to abort the previous stream when the user sends a second message merges two assistants. Enforce a single in-flight request with the isStreaming guard.

Model routing. Hardcoding model in the composable limits reuse. Pass it as a parameter to send or make it a ref so the UI can switch models without reinitializing.

Cache hints. If your gateway forwards provider cache-control hints, you can append a system message with static context to hit prompt caches. The composable stays unchanged; just structure messages wisely.

9. Production hardening checklist

• Add request timeouts via AbortSignal.timeout in addition to manual stop.
• Sanitize message content before rendering to avoid HTML injection if you use v-html.
• Persist messages to localStorage with a watcher, but strip incomplete assistant messages on reload.
• Expose tokenUsage if your endpoint returns it; n4n.ai meters per-token usage, which you can surface in the UI for cost tracking.

The vue composable llm streaming chat pattern separates transport concerns from UI. Once this primitive is solid, building multi-turn agents, tool calls, or branching conversations becomes a matter of composing refs rather than wrestling with fetch.