Debugging LLM timeouts on Vercel serverless functions

A llm timeout vercel serverless function typically surfaces as a 504 from Vercel’s gateway when your route blocks on a model response past the platform’s execution cap. The default hobby limit is 10 seconds; pro plans allow up to 60 seconds per function. Most fixes come from eliminating blocking waits, not from throwing more compute at the problem.

Step 1: Reproduce the timeout with your exact deployment config

Vercel’s local vercel dev does not enforce maxDuration, so you must set it explicitly and test against the real constraint. Create a minimal API route that mirrors your production call.

// app/api/llm/route.ts
export const maxDuration = 10; // hobby plan limit

export async function POST(req: Request) {
  const { prompt } = await req.json();
  const start = Date.now();
  const res = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.OPENAI_KEY}` },
    body: JSON.stringify({ model: "gpt-4o-mini", messages: [{ role: "user", content: prompt }] }),
  });
  const data = await res.json();
  return Response.json({ latency: Date.now() - start, data });
}

Deploy this to a preview environment and hit it with a slow prompt. If you see FUNCTION_INVOCATION_TIMEOUT, you have reproduced the issue. The key is to confirm the timeout is the platform killing the function, not an SDK error.

Step 2: Instrument the LLM call with hard timing boundaries

You cannot fix what you cannot measure. Wrap the outbound request in an AbortController and log phases: DNS, connect, first byte, full body.

export async function POST(req: Request) {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), 8000);
  const t0 = performance.now();
  try {
    const res = await fetch("https://api.openai.com/v1/chat/completions", {
      method: "POST",
      signal: controller.signal,
      headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.OPENAI_KEY}` },
      body: JSON.stringify({ model: "gpt-4o-mini", messages: [{ role: "user", content: "explain recursion" }] }),
    });
    const ttfb = performance.now() - t0;
    const data = await res.json();
    console.log(JSON.stringify({ ttfb_ms: Math.round(ttfb), total_ms: Math.round(performance.now() - t0) }));
    return Response.json(data);
  } catch (e) {
    console.error("llm call failed", e.name, Math.round(performance.now() - t0));
    throw e;
  } finally {
    clearTimeout(timeout);
  }
}

Check Vercel’s function logs. If ttfb_ms is 7.5s and total is 9s, the model is slow to start. That confirms the llm timeout vercel serverless function is caused by provider latency, not your code.

Step 3: Convert the blocking call to a streaming response

Streaming moves the first token to the client in hundreds of milliseconds and keeps the connection open within the duration limit. In Next.js App Router, return a ReadableStream.

export const maxDuration = 60;

export async function POST(req: Request) {
  const res = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.OPENAI_KEY}` },
    body: JSON.stringify({
      model: "gpt-4o-mini",
      stream: true,
      messages: [{ role: "user", content: "write a haiku" }],
    }),
  });

  const reader = res.body!.getReader();
  const stream = new ReadableStream({
    async start(controller) {
      while (true) {
        const { done, value } = await reader.read();
        if (done) break;
        controller.enqueue(value);
      }
      controller.close();
    },
  });

  return new Response(stream, { headers: { "Content-Type": "text/event-stream" } });
}

Streaming does not eliminate the total time cap, but it prevents the client from hanging and lets you push partial results. For a 60-second pro function, a streaming 40-second generation is survivable; a blocking one is not.

Step 4: Add fallback routing to absorb provider degradation

When a single provider is slow, a timeout is inevitable. Route through a layer that retries or fails over. If you route through an OpenRouter-class gateway such as n4n.ai, it honors client routing directives and automatically falls back when a provider is rate-limited, which directly cuts llm timeout vercel serverless function errors. You can also implement a simple secondary fetch with a shorter timeout:

async function callWithFallback(prompt: string) {
  const primary = fetch("https://api.openai.com/v1/chat/completions", { /* ... */ });
  const secondary = fetch("https://api.anthropic.com/v1/messages", { /* ... */ });
  const winner = await Promise.any([primary, secondary]);
  return winner;
}

Use Promise.any with per-request AbortControllers set to 5s each. The first successful stream wins; the loser is cancelled. This pattern converts a hard timeout into a latency spike.

Step 5: Offload long jobs to a background worker when limits are exceeded

If your use case needs a 5-minute summarization, Vercel serverless is the wrong place to block. Push the job to a queue (Upstash QStash, Inngest, or a simple DynamoDB poll) and return a job ID.

// api/start/route.ts
export async function POST(req: Request) {
  const jobId = crypto.randomUUID();
  await fetch("https://qstash.upstash.io/v1/publish", {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.QSTASH_TOKEN}`, "Upstash-Forward-Url": "https://your-app.vercel.app/api/worker" },
    body: JSON.stringify({ jobId, prompt: "long doc" }),
  });
  return Response.json({ jobId, status: "queued" });
}

The worker function still has a duration limit, but it runs after the client disconnects. Poll the job status from the client via a separate lightweight endpoint. This removes the llm timeout vercel serverless function class of errors entirely for long tasks.

Step 6: Verify success with targeted load and log checks

Verification is concrete. First, curl the streaming endpoint with -N and time it:

curl -N -X POST https://your-app.vercel.app/api/llm \
  -H "Content-Type: application/json" \
  -d '{"prompt":"hello"}' \
  -w "total_time:%{time_total}\n"

You should see tokens arrive incrementally and time_total under your maxDuration. Second, check Vercel logs for zero FUNCTION_INVOCATION_TIMEOUT entries over 100 requests. Third, simulate provider slowdown with a proxy that delays responses by 9s; your fallback or streaming should keep the function alive.

If the logs show sub-2s TTFB and continuous streams, the debugging is done. The llm timeout vercel serverless function problem is solved by architecture, not by config tweaks.

Extra: Set cache-control to avoid repeated slow calls

Providers and gateways forward cache hints. If your prompt is deterministic, send Cache-Control: max-age=3600 to the model endpoint or gateway. n4n.ai forwards provider cache-control hints, so repeated identical requests hit cache instead of recomputing. This shrinks tail latency and keeps you away from the timeout cliff.

{
  "headers": { "Cache-Control": "max-age=3600" }
}

Apply that header on the fetch call in Step 3. Measure again; repeated calls should drop to milliseconds.

Closing checklist

• Reproduced timeout in preview with correct maxDuration.
• Added AbortController and phase logging.
• Switched to streaming for any response >5s.
• Implemented fallback or used a gateway with automatic provider switch.
• Moved >60s jobs to background workers.
• Verified with curl -N and Vercel log scan.

Follow these steps and the llm timeout vercel serverless function error stops being a mystery and becomes a solved constraint.