How to add rate limiting to a Next.js AI chat API route

A naive next.js rate limiting ai chat api route will happily forward every request to your LLM provider until you get a surprise bill or a 429 from upstream. In this guide we’ll harden an App Router chat endpoint with per-IP rate limits using Upstash Redis and the Vercel AI SDK, so you control throughput and cost before any tokens are generated.

We’ll build a production-grade next.js rate limiting ai chat api route that returns proper 429 headers, works on Vercel’s edge runtime, and degrades gracefully under load.

Step 1: Create the baseline chat route

Start with a minimal App Router route that streams chat completions using the Vercel AI SDK. This is the surface we’re going to protect.

// app/api/chat/route.ts
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export const runtime = 'edge';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });

  return result.toDataStreamResponse();
}

This works, but it has no guardrails. Anyone who finds the endpoint can loop requests indefinitely.

Step 2: Install the rate limiting stack

We’ll use Upstash Redis because it has a REST API that runs cleanly on the edge runtime without a persistent socket. The @upstash/ratelimit package implements sliding-window and token-bucket algorithms with zero boilerplate.

pnpm add @upstash/ratelimit @upstash/redis

Set the following environment variables in .env.local (and in your Vercel project):

UPSTASH_REDIS_REST_URL=https://your-instance.upstash.io
UPSTASH_REDIS_REST_TOKEN=your-token

Step 3: Initialize the ratelimit client

Create a small module that constructs the limiter. A sliding window of 10 requests per minute is a sane default for an anonymous chat endpoint.

// lib/ratelimit.ts
import { Redis } from '@upstash/redis';
import { Ratelimit } from '@upstash/ratelimit';

const redis = new Redis({
  url: process.env.UPSTASH_REDIS_REST_URL!,
  token: process.env.UPSTASH_REDIS_REST_TOKEN!,
});

export const chatRatelimit = new Ratelimit({
  redis,
  limiter: Ratelimit.slidingWindow(10, '1 m'),
  prefix: 'chat-ratelimit',
  analytics: false,
});

The prefix isolates this limit from any other rate limits you might run in the same Redis instance.

Step 4: Identify the caller and enforce the limit

In a serverless environment, the real client IP is forwarded in the x-forwarded-for header. Extract the first address, fall back to a constant for local dev, and run the limit check before you spend any LLM tokens.

// app/api/chat/route.ts
import { chatRatelimit } from '@/lib/ratelimit';
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';

export const runtime = 'edge';

export async function POST(req: Request) {
  const ip =
    req.headers.get('x-forwarded-for')?.split(',')[0]?.trim() ?? '127.0.0.1';

  const { success, limit, remaining, reset } =
    await chatRatelimit.limit(ip);

  if (!success) {
    return new Response('Too many requests', {
      status: 429,
      headers: {
        'Retry-After': String(Math.ceil((reset - Date.now()) / 1000)),
        'X-RateLimit-Limit': String(limit),
        'X-RateLimit-Remaining': String(remaining),
      },
    });
  }

  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
  });

  return result.toDataStreamResponse();
}

If you have authentication, replace ip with a user ID from your session. Rate limiting on a stable user identifier is strictly better than IP because it survives NAT and VPN pooling.

const userId = (await getSession(req))?.user?.id ?? ip;
const { success } = await chatRatelimit.limit(userId);

Step 5: Return structured 429 responses

The code above already sets Retry-After and rate-limit headers. Clients like the Vercel AI SDK’s useChat hook will surface the 429, but you should also return a JSON body if your frontend expects it.

if (!success) {
  return new Response(
    JSON.stringify({
      error: 'rate_limit_exceeded',
      retryAfter: Math.ceil((reset - Date.now()) / 1000),
    }),
    {
      status: 429,
      headers: {
        'content-type': 'application/json',
        'retry-after': String(Math.ceil((reset - Date.now()) / 1000)),
      },
    }
  );
}

Do the check before calling req.json() on large payloads—no point parsing a 50-message transcript if you’re going to reject the request anyway.

Step 6: Connect an LLM provider with upstream resilience

The route now blocks abuse at the edge. The next concern is upstream provider reliability. If you point the SDK at an OpenAI-compatible gateway such as n4n.ai, which fronts 240+ models and automatically fails over when a provider is rate-limited or degraded, your own rate limit still applies first, but you avoid cascading 429s from a single vendor.

import { createOpenAI } from '@ai-sdk/openai';

const openai = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY,
});

// inside POST:
const result = streamText({
  model: openai('gpt-4o-mini'),
  messages,
});

This keeps the next.js rate limiting ai chat api route in full control of client-facing throttling while outsourcing model routing and per-token metering to the gateway.

Step 7: Verify the rate limit end to end

Deploy to Vercel or run pnpm dev and hammer the endpoint with a loop.

for i in {1..15}; do
  curl -s -o /dev/null -w "%{http_code}\n" \
    -X POST http://localhost:3000/api/chat \
    -H 'content-type: application/json' \
    -d '{"messages":[{"role":"user","content":"hi"}]}'
done

The first 10 requests should return 200; the remaining should return 429 with retry-after populated. Check headers explicitly:

curl -i -X POST http://localhost:3000/api/chat \
  -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"hi"}]}'

Look for x-ratelimit-limit: 10 and x-ratelimit-remaining decrementing on each call.

Tuning limits for production

Ten requests per minute per IP is strict but safe for a public demo. For authenticated users, bump to 60–100 per minute and consider a token-weighted limit (count input+output tokens instead of requests) if your prompts are large. @upstash/ratelimit supports a tokenBucket limiter if you want to allow short bursts:

limiter: Ratelimit.tokenBucket(50, '1 m', 10), // 50 tokens/min, burst of 10

Remember that rate limiting is not auth. It’s a cost and abuse control. Pair it with CORS restrictions and, ideally, a signed session before you ship.

Edge runtime caveats

Upstash Redis uses HTTPS, so it works on runtime = 'edge'. If you switch to Node runtime for longer timeouts, the same code works but you lose the global low-latency edge benefit. Never use an in-memory Map for rate limiting on Vercel—serverless instances are ephemeral and scale to zero, so the counter will reset constantly and provide no protection.

A correctly implemented next.js rate limiting ai chat api route is a few dozen lines of code and the difference between a sustainable LLM feature and a runaway bill. Ship the limit before you ship the model.