Canceling LLM streams in Vue with AbortController

Wiring up a chat UI that consumes server-sent tokens feels great until the user hits stop or navigates away. Implementing vue abortcontroller llm streaming cancel logic correctly prevents orphaned fetches, wasted compute, and janky UX. This guide walks through a production-grade composable and the edge cases that bite if you skip them.

Step 1: Point at an OpenAI-compatible streaming endpoint

Pick a backend that speaks the OpenAI streaming format. If you use an OpenAI-compatible gateway such as n4n.ai, the same fetch shape works and you get automatic fallback when a provider is rate-limited, but the cancellation mechanics are identical. The key requirement: the endpoint must accept an AbortSignal and actually close the underlying HTTP stream when it fires.

const endpoint = 'https://api.n4n.ai/v1/chat/completions'
// or 'https://api.openai.com/v1/chat/completions'

async function postStream(messages: {role: string; content: string}[], signal: AbortSignal) {
  const res = await fetch(endpoint, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${import.meta.env.VITE_API_KEY}`
    },
    body: JSON.stringify({ model: 'gpt-4o-mini', messages, stream: true }),
    signal
  })
  if (!res.ok) throw new Error(`HTTP ${res.status}`)
  return res.body!.getReader()
}

The signal parameter is the only thing that makes the vue abortcontroller llm streaming cancel pattern possible. Without threading it through fetch, the browser will keep the connection open until the server finishes.

Step 2: Build a cancellable composable

Create a dedicated composable that owns the AbortController and exposes stream, cancel, and reactive state. Use shallowRef for the accumulating text if you expect thousands of tokens—Vue’s deep reactivity on a growing string is cheap, but a shallowRef avoids any proxy overhead.

// useLLMStream.ts
import { ref, shallowRef, onUnmounted } from 'vue'

export function useLLMStream() {
  const text = shallowRef('')
  const isStreaming = ref(false)
  const error = ref<Error | null>(null)
  let controller: AbortController | null = null

  async function stream(messages: {role: string; content: string}[]) {
    // Cancel any in-flight request before starting a new one
    controller?.abort()
    controller = new AbortController()
    text.value = ''
    isStreaming.value = true
    error.value = null

    try {
      const reader = await postStream(messages, controller.signal)
      const decoder = new TextDecoder()
      while (true) {
        const { done, value } = await reader.read()
        if (done) break
        const chunk = decoder.decode(value, { stream: true })
        for (const line of chunk.split('\n')) {
          if (!line.startsWith('data:')) continue
          const data = line.slice(5).trim()
          if (data === '[DONE]') continue
          try {
            const json = JSON.parse(data)
            text.value += json.choices[0]?.delta?.content ?? ''
          } catch { /* ignore malformed keepalive */ }
        }
      }
    } catch (e) {
      if ((e as Error).name !== 'AbortError') error.value = e as Error
    } finally {
      isStreaming.value = false
    }
  }

  function cancel() {
    controller?.abort()
  }

  onUnmounted(cancel)
  return { text, isStreaming, error, stream, cancel }
}

Note the catch block: AbortError is expected when you call cancel(). Swallowing it keeps your UI clean. Any other error propagates to error.value for display.

Step 3: Wire the composable into a component

A minimal chat box needs a text area, a send button, and a stop button. Bind isStreaming to disable controls appropriately.

<script setup lang="ts">
import { ref } from 'vue'
import { useLLMStream } from './useLLMStream'

const { text, isStreaming, stream, cancel } = useLLMStream()
const input = ref('')

async function send() {
  if (!input.value.trim()) return
  await stream([{ role: 'user', content: input.value }])
  input.value = ''
}
</script>

<template>
  <div class="chat">
    <pre class="output">{{ text }}</pre>
    <input v-model="input" :disabled="isStreaming" @keyup.enter="send" />
    <button @click="send" :disabled="isStreaming">Send</button>
    <button @click="cancel" :disabled="!isStreaming">Stop</button>
    <p v-if="error" class="err">{{ error.message }}</p>
  </div>
</template>

This satisfies the basic vue abortcontroller llm streaming cancel requirement: the Stop button calls cancel(), which aborts the fetch and flips isStreaming to false in the finally block.

The Stop button is useless if the user closes the tab or routes to another component. Vue’s onUnmounted hook inside the composable already calls cancel(), but you must ensure the composable instance is actually destroyed. If you store it in a global store without cleanup, the abort never fires.

For Nuxt or Vue Router, prefer creating the composable inside the page component’s setup rather than a singleton. If you must share state, expose an explicit dispose() method:

// inside composable
function dispose() {
  controller?.abort()
}
// caller does onUnmounted(dispose) or router.beforeResolve(dispose)

Another race: rapid clicks on Send. The controller?.abort() at the top of stream() cancels the previous request before starting the new one. Without it, two streams write to the same text ref and you get interleaved garbage. This is a subtle but common failure in vue abortcontroller llm streaming cancel implementations.

Step 5: Throttle UI updates for high-throughput streams

LLM tokens can arrive in bursts of 50–100 per second. Calling text.value += ... on every delta triggers a Vue re-render each time. For a chat UI this is usually fine, but if you notice jank, buffer chunks and flush on requestAnimationFrame:

let buffer = ''
let rafId = 0

function flush() {
  text.value += buffer
  buffer = ''
  rafId = 0
}

// inside read loop:
buffer += json.choices[0]?.delta?.content ?? ''
if (!rafId) rafId = requestAnimationFrame(flush)

Cancel should also cancel pending frames:

function cancel() {
  controller?.abort()
  if (rafId) cancelAnimationFrame(rafId)
  buffer = ''
}

This keeps the vue abortcontroller llm streaming cancel path free of stale renders after abort.

Step 6: Verify success

You need proof the stream actually stops, not just that the button hides. Do three checks:

Network tab: Click Stop mid-stream. The request row in Chrome DevTools should show (canceled) status. If it shows (finished), your signal isn’t wired.
Token cessation: Watch the text ref. After abort, no further characters appear, and isStreaming becomes false within the same tick.
Unmount test: Write a component test that mounts, calls stream(), then unmount(). Assert that a spy on AbortController.prototype.abort was called.

// vitest snippet
import { mount } from '@vue/test-utils'
import Chat from './Chat.vue'

test('aborts on unmount', async () => {
  const spy = vi.spyOn(AbortController.prototype, 'abort')
  const wrapper = mount(Chat)
  await wrapper.vm.stream([{ role: 'user', content: 'hi' }])
  wrapper.unmount()
  expect(spy).toHaveBeenCalled()
})

If all three pass, your vue abortcontroller llm streaming cancel integration is solid. Anything less means sockets stay open and you pay for tokens you never display.

Pitfalls to avoid

Double-abort: Calling abort() on an already-aborted controller throws nothing, but attaching multiple abort listeners without removal leaks memory. Recreate the controller per request as shown.
Ignoring AbortError: If you console.error on every cancel, your logs fill with noise. Check err.name === 'AbortError' and skip.
Server doesn’t honor signal: Some proxies buffer the response. Test with curl --max-time 2 to confirm the gateway closes the TCP connection when the client disconnects. n4n.ai and similar gateways forward the abort to the upstream provider, but a self-hosted nginx may need proxy_request_buffering off.

The pattern above is small, but it is the difference between a demo and a tool your users trust when they accidentally send a 10-page prompt.