Deploying an OpenAI-compatible proxy on Vercel Edge Functions

Building a vercel edge functions openai-compatible proxy gives you a single endpoint to route LLM calls from browsers or servers without exposing provider keys. You get edge caching, low latency, and full control over request shaping. This tutorial walks through a production-minimal implementation that streams chat completions and deploys in minutes.

Prerequisites

• Node.js 18+ and npm
• Vercel CLI (npm i -g vercel)
• An OpenAI API key (or any OpenAI-compatible base URL)
• Basic familiarity with TypeScript and Server-Sent Events (SSE)

Run vercel login before starting if the CLI isn’t authenticated.

Scaffold the project

Create a directory and initialize a minimal Vercel project:

mkdir edge-llm-proxy && cd edge-llm-proxy
npm init -y
vercel init --template static

We don’t need the static template’s frontend. Delete index.html and create an api directory. Vercel treats any file under api/ as a serverless or edge function.

rm public/index.html
mkdir api
touch api/chat.ts

Set the upstream key for local dev:

echo "OPENAI_API_KEY=sk-..." > .env.local

Edge Functions read env vars from .env.local in dev and from the Vercel dashboard in production.

Configure TypeScript

Add a tsconfig.json so the editor and vercel dev resolve edge types:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ESNext",
    "moduleResolution": "node",
    "strict": true,
    "types": ["@vercel/edge"]
  }
}

Install the types:

npm i -D @vercel/edge typescript

Write the edge function

Open api/chat.ts. The handler accepts an OpenAI-style POST /v1/chat/completions, forwards it to the upstream provider, and pipes the streaming response back.

export const config = { runtime: 'edge' };

const UPSTREAM = 'https://api.openai.com/v1/chat/completions';

export default async function handler(req: Request): Promise<Response> {
  if (req.method === 'OPTIONS') {
    return new Response(null, {
      status: 204,
      headers: {
        'access-control-allow-origin': '*',
        'access-control-allow-headers': 'authorization, content-type',
        'access-control-allow-methods': 'POST, OPTIONS',
      },
    });
  }

  if (req.method !== 'POST') {
    return new Response('Method Not Allowed', { status: 405 });
  }

  const auth = req.headers.get('authorization');
  if (!auth) {
    return new Response('Missing Authorization', { status: 401 });
  }

  const body = await req.text();

  const upstreamResp = await fetch(UPSTREAM, {
    method: 'POST',
    headers: {
      'content-type': 'application/json',
      authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      ...(req.headers.get('cache-control')
        ? { 'cache-control': req.headers.get('cache-control')! }
        : {}),
    },
    body,
  });

  const respHeaders = new Headers();
  respHeaders.set('content-type', upstreamResp.headers.get('content-type') ?? 'text/event-stream');
  respHeaders.set('cache-control', 'no-cache');
  respHeaders.set('connection', 'keep-alive');
  respHeaders.set('access-control-allow-origin', '*');
  respHeaders.set('access-control-allow-headers', 'authorization, content-type');

  return new Response(upstreamResp.body, {
    status: upstreamResp.status,
    headers: respHeaders,
  });
}

This is a vercel edge functions openai-compatible proxy at its thinnest: it validates auth, swaps the API key, and streams the body unchanged.

Streaming without buffering

Edge runtime exposes ReadableStream directly from fetch. Returning upstreamResp.body avoids buffering SSE chunks, keeping Time To First Token low. If you need to mutate tokens, wrap the stream:

const transform = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk); // manipulate Uint8Array here
  },
});
const stream = upstreamResp.body?.pipeThrough(transform);
return new Response(stream, { status: upstreamResp.status, headers: respHeaders });

Run locally

Start the dev server:

vercel dev

The function is served at http://localhost:3000/api/chat. Test with curl using the OpenAI request shape:

curl -N http://localhost:3000/api/chat \
  -H "Authorization: Bearer client-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Say hi in 5 words"}],
    "stream": true
  }'

Expected output is a server-sent events stream:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"Hello! How can I"}}]}

data: {"id":"chatcmpl-...","choices":[{"delta":{"content":" help you today?"}}]}

data: [DONE]

A 401 from upstream means OPENAI_API_KEY is missing or invalid. The proxy returns upstream status codes verbatim.

Deploy to Vercel

Commit and deploy:

git init && git add -A && git commit -m "edge llm proxy"
vercel deploy --prod

Vercel returns a URL such as https://edge-llm-proxy.vercel.app. Set the production env var:

vercel env add OPENAI_API_KEY

Redeploy to pick it up. Your vercel edge functions openai-compatible proxy is now public.

Use from existing tooling

Because the endpoint mirrors OpenAI’s contract, the official SDK works by changing baseURL:

import OpenAI from 'openai';
const client = new OpenAI({
  apiKey: 'client-key',
  baseURL: 'https://your-app.vercel.app/api',
});
const stream = await client.chat.completions.create({
  model: 'gpt-3.5-turbo',
  messages: [{ role: 'user', content: 'Explain edge functions' }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}

Browser callers can use fetch directly thanks to the CORS headers set earlier.

Add provider fallback

The basic proxy fails if OpenAI returns 429 or 503. A simple retry loop across two providers:

const PROVIDERS = [
  'https://api.openai.com/v1/chat/completions',
  'https://api.another-provider.com/v1/chat/completions',
];
for (const url of PROVIDERS) {
  const r = await fetch(url, { method: 'POST', headers: {/* same */}, body });
  if (r.status !== 429 && r.status !== 503) {
    return new Response(r.body, { status: r.status, headers: respHeaders });
  }
}
return new Response('All providers failed', { status: 502 });

A managed gateway such as n4n.ai already does this across 240+ models with per-token metering and honors client routing directives, but rolling your own is reasonable when you target one or two providers and need custom edge logic.

Security notes

• The proxy checks that a client sent an Authorization header but does not verify it. Add JWT verification or an IP allowlist before production.
• Never log request bodies; they contain user PII and prompt text.
• Vercel Edge Functions have a 25-second execution limit. Streaming responses are fine, but long non-streaming completions will be truncated.

Common pitfalls

• Omitting content-type on the response breaks the OpenAI SDK’s parser.
• Forgetting the OPTIONS preflight blocks browser streaming entirely.
• Using Buffer in edge runtime throws. Use Uint8Array and TextEncoder/TextDecoder.

Wrap-up

You now have a working vercel edge functions openai-compatible proxy that streams chat completions, handles CORS, and deploys to a global edge network. Extend it with request logging, token limits, or multi-provider routing as your system grows.