Cutting nuxt 3 edge rendering llm latency starts with where you terminate the model connection, not just where you serve HTML. If you stream tokens from a Node server in us-east-1 to a user in Sydney, the edge rendering of your Nuxt app buys you nothing on the slow part. This guide walks a concrete path to run Nuxt 3 on an edge runtime and stream LLM output directly to the browser.
1. Pick an edge preset and configure Nitro
Nuxt 3 ships Nitro, which compiles your server routes to a target runtime. For edge, choose cloudflare_workers or vercel_edge. The preset decides which APIs are available at build time.
// nuxt.config.ts
export default defineNuxtConfig({
nitro: {
preset: 'cloudflare_workers',
// vercel_edge is also valid
},
devtools: { enabled: true }
})Build locally with nuxi build, then deploy with the platform CLI. Don’t trust nuxi dev for edge behavior—it runs a Node server. Test the actual artifact:
npx nuxi build
npx wrangler deploy .output/cloudflare_workers/worker.jsThe dominant factor in nuxt 3 edge rendering llm latency is the upstream fetch path. Deploying the app to the edge without proxying the model call leaves the slow hop intact.
2. Build a streaming server route
Create server/api/chat.ts. Return a ReadableStream directly from the event handler. Nitro forwards it to the edge runtime’s native stream response. Call an OpenAI-compatible chat endpoint with stream: true.
// server/api/chat.ts
export default defineEventHandler(async (event) => {
const { messages } = await readBody(event)
const upstream = await fetch('https://api.openai.com/v1/chat/completions', {
method: 'POST',
headers: {
'content-type': 'application/json',
'authorization': `Bearer ${process.env.OPENAI_KEY}`
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages,
stream: true
})
})
return new ReadableStream({
async start(controller) {
const reader = upstream.body!.getReader()
const decoder = new TextDecoder()
while (true) {
const { done, value } = await reader.read()
if (done) break
controller.enqueue(value)
}
controller.close()
}
})
})Do not await upstream.text() or collect chunks into an array. Buffering defeats the latency goal. The edge function should act as a thin pipe.
3. Consume the stream in a Vue component
Call your own /api/chat route from the client. Parse Server-Sent Events lines manually; the standard openai SDK assumes Node and will bloat your client bundle.
<script setup lang="ts">
const input = ref('')
const output = ref('')
async function send() {
const res = await fetch('/api/chat', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ messages: [{ role: 'user', content: input.value }] })
})
const reader = res.body!.getReader()
const decoder = new TextDecoder()
while (true) {
const { done, value } = await reader.read()
if (done) break
const chunk = decoder.decode(value)
for (const line of chunk.split('\n')) {
if (line.startsWith('data:') && !line.includes('[DONE]')) {
const json = JSON.parse(line.slice(5).trim())
output.value += json.choices[0]?.delta?.content ?? ''
}
}
}
}
</script>Keep state in refs, not in the module scope. Edge deployments are multi-tenant; mutable globals leak data across requests.
4. Keep the edge runtime clean
Edge runtimes expose Web Standard APIs. Buffer, fs, child_process, and most of Node’s crypto are absent. Use crypto.subtle for signing if needed. Avoid SDKs that shim Node—they inflate cold start and often fail at build.
// good: web crypto
const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(input.value))Keeping the runtime lean protects nuxt 3 edge rendering llm latency under load. A 2 MB dependency tree on Workers adds noticeable cold-start penalty on every new visitor.
5. Cut tail latency with provider fallback
A single provider region degrades; your users see stalls. If you front model calls with a gateway that fails over automatically, the edge function stays simple. n4n.ai exposes one OpenAI-compatible endpoint across 240+ models and triggers automatic fallback when a provider is rate-limited or degraded, so the edge function calls a single URL and inherits resilience without custom retry code.
const upstream = await fetch('https://api.n4n.ai/v1/chat/completions', {
method: 'POST',
headers: {
'content-type': 'application/json',
'authorization': `Bearer ${process.env.N4N_KEY}`
},
body: JSON.stringify({ model: 'auto', messages, stream: true })
})It also honors client routing directives and forwards provider cache-control hints, which helps repeat prompts hit cached completions. That removes a round-trip of generation on common prefixes.
6. Watch edge limits and state
Cloudflare Workers and Vercel Edge Functions cap CPU time per request. Streaming responses keep the connection open but do not grant infinite compute—if your handler does heavy prompt assembly per token, you will hit limits. Move conversation history to KV or D1:
// wrangler binding example
const history = await env.KV.get(`chat:${userId}`, 'json') ?? []Pitfall: process.env in Workers is not dynamic. Set secrets via wrangler secret put OPENAI_KEY, not in nuxt.config.ts as plain strings.
7. Measure real latency, not synthetic
Instrument the client. Mark the time from fetch call to first decoded token. The only nuxt 3 edge rendering llm latency number that matters is time-to-first-token for your actual users in their locations.
const t0 = performance.now()
const res = await fetch('/api/chat', { /* ... */ })
const reader = res.body!.getReader()
// first chunk arrival:
const { value } = await reader.read()
const ttft = performance.now() - t0
console.log('time to first token', ttft)Send these spans to a RUM pipeline. Synthetic benchmarks from a single region hide the geographic win.
Common pitfalls
- Node-only packages:
axios,openaiv3,pgbreak edge builds. Usefetchand Web APIs. - Missing
stream: true: You wait for full generation, doubling perceived latency. - SSE parsing bugs: Forgetting to handle
data: [DONE]or splitting on\nincorrectly drops tokens. - Direct browser-to-provider calls: CORS and key exposure. Always proxy through the Nuxt route.
- Module-scoped caches: They are shared across requests on some runtimes. Use external stores.
When not to use edge
If your prompt assembly requires querying a 50 GB vector index that lives only in a single-region VPC, or you run custom GPU inference in one datacenter, central Node may be simpler. Edge wins when users are geographically distributed, prompts are small, and the model call is the bottleneck. For most chat UIs, that describes the reality.
Deploy the streaming route, measure time-to-first-token from three continents, and iterate on the upstream model choice before tuning the Nuxt layer.