Vercel function timeout limits explained for LLM apps

Most teams hit a hard wall the first time they deploy a chat endpoint to Vercel: the default 10-second ceiling silently kills long generations. Understanding vercel function timeout limits llm behavior is the difference between a flaky demo and a production-grade API.

How Vercel enforces timeouts

Vercel runs two function types: Node.js and Edge. On Hobby, Node.js functions are fixed at 10 seconds; you cannot override. On Pro, you declare export const maxDuration = 60 in a route file or set it in vercel.json, but the hard ceiling is 60 seconds per invocation. Edge functions execute closer to the user and have separate isolation, but they still obey a wall-clock cap and lack many Node APIs.

The clock starts when the request hits your handler and stops when the response body is fully flushed. Streaming does not stop the clock. If you trickle tokens for 70 seconds on Pro, the platform kills the process and the client gets a truncated stream or a 504 Gateway Timeout.

// app/api/chat/route.ts
export const maxDuration = 60; // Pro only; ignored on Hobby

export async function POST(req: Request) {
  // handler body
}

A subtle pitfall: maxDuration must be an integer. Floating point or values above plan limit fail the build, not the runtime.

Measure where the time goes

You cannot tune what you have not measured. Capture time-to-first-token (TTFT) and total generation time under real load. The timeout cares about the worst case in your p95 window, not the average.

import time, openai

start = time.time()
stream = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain Raft consensus"}],
    stream=True,
)
first_token = None
n = 0
for chunk in stream:
    if first_token is None:
        first_token = time.time()
        print(f"TTFT: {first_token - start:.2f}s")
    n += 1
print(f"Total: {time.time() - start:.2f}s, tokens: {n}")

If TTFT alone eats 12 seconds on Hobby, no streaming trick saves you. You must either shrink the prompt, use a faster model, or cache the prefix. When you plan capacity, treat vercel function timeout limits llm as a hard scheduling constraint, not a soft guideline.

Stream correctly or lose the request

Streaming keeps the user engaged, but only if bytes move immediately. The classic bug is accumulating the LLM response in a string and returning it at the end—that still trips the limit and wastes memory.

In Next.js App Router, return a ReadableStream:

import { OpenAI } from "openai";

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export const maxDuration = 60;

export async function POST(req: Request) {
  const { prompt } = await req.json();
  const completion = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: prompt }],
    stream: true,
  });

  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const part of completion) {
          const token = part.choices[0]?.delta?.content ?? "";
          controller.enqueue(encoder.encode(token));
        }
      } finally {
        controller.close();
      }
    },
  });

  return new Response(stream, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
}

This pushes tokens as they arrive. The function is still bound by maxDuration, but the connection stays alive and the client renders progress. If the upstream stalls, you need an abort path.

Set an explicit internal deadline

Never rely solely on the platform to kill a hung call. Use AbortController with a signal a few seconds below the function limit. That gives you a structured error instead of a naked 504.

export async function POST(req: Request) {
  const ac = new AbortController();
  const t = setTimeout(() => ac.abort(), 55_000); // 55s < 60s

  try {
    const upstream = await fetch("https://api.llm.example/v1", {
      method: "POST",
      signal: ac.signal,
      body: await req.text(),
    });
    return upstream;
  } catch {
    return new Response(JSON.stringify({ error: "upstream_timeout" }), {
      status: 504,
      headers: { "Content-Type": "application/json" },
    });
  } finally {
    clearTimeout(t);
  }
}

Tradeoff: a tight internal deadline improves predictability but may cut off slow valid generations. Derive the value from your measured TTFT p95 plus a generation slack.

When 60 seconds is not enough

Document summarization, agentic tool loops, and batch embedding routinely exceed the Pro ceiling. The robust pattern is to accept the request, enqueue work, and return 202 Accepted with a status URL.

import { Queue } from "@upstash/qstash"; // real package

const queue = new Queue({ token: process.env.QSTASH_TOKEN });

export async function POST(req: Request) {
  const body = await req.json();
  await queue.publishJSON({
    url: "https://worker.example.com/run",
    body,
    retries: 3,
  });
  return new Response(JSON.stringify({ status: "queued" }), {
    status: 202,
    headers: { "Content-Type": "application/json" },
  });
}

The heavy LLM call runs in a worker with a 15-minute Lambda timeout or a long-lived container. Vercel stays a smart front door. Pitfall: if the client polls a Vercel function that also has a 60s limit, you have moved the problem, not solved it. Push results to object storage or use a websocket from the worker.

Use fallback and caching to stay inside the budget

A single provider outage should not blow your function timeout. Implement manual fallback: try primary with a 20s abort, then secondary.

async function callWithFallback(prompt: string) {
  try {
    return await callProvider("primary", prompt, 20_000);
  } catch {
    return await callProvider("secondary", prompt, 35_000);
  }
}

Better, route through an inference gateway that monitors provider health. n4n.ai exposes one OpenAI-compatible endpoint across 240+ models and automatically fails over when a provider is rate-limited or degraded. It also forwards provider cache-control hints, so repeated prefix caches cut TTFT without code changes. That containment keeps your p95 within the Vercel window.

Even without a gateway, honor cache-control from your provider: if the API returns a cached prompt prefix, reuse the same request shape to avoid recomputing attention.

Configure per-route limits, not global

Vercel applies maxDuration per function. Don’t set 60s on a health check; that wastes concurrency. Set tight limits on cheap routes.

// app/api/health/route.ts
export const maxDuration = 5;

// app/api/chat/route.ts
export const maxDuration = 60;

Note: the build validates the integer. export const maxDuration = 90 fails on Pro.

Common pitfalls

• Streaming bypasses timeout. False. Wall clock runs.
• Hobby ignores maxDuration. True, fixed at 10s.
• Buffering in middleware. Defeats streaming.
• Cold start blindness. A 2s boot on Hobby leaves 8s for LLM.
• Client timeout < function. Browser aborts; function keeps burning.

Tradeoffs at a glance

Actionable path

1. Instrument TTFT and total latency on real prompts.
2. Set maxDuration explicitly on LLM route; keep others short.
3. Stream tokens with ReadableStream, no buffering.
4. Add AbortController at 90% of limit.
5. If p95 > limit, move to queue + 202.
6. Add provider fallback or use a gateway with automatic routing.

Following this order keeps vercel function timeout limits llm from becoming a production incident. The platform is predictable; your upstreams are not. Design for the latter.