Building a Cloudflare Worker proxy for OpenAI-compatible APIs

You can put a thin edge layer in front of any LLM endpoint that speaks the OpenAI protocol. This guide walks through building a cloudflare worker proxy openai-compatible service that forwards chat completions, injects credentials, and streams tokens without buffering.

Prerequisites

• Node.js 18+ and Wrangler installed (npm i -g wrangler)
• A Cloudflare account with Workers enabled
• An API key for an OpenAI-compatible backend (OpenAI, or a gateway such as n4n.ai which exposes one endpoint for 240+ models with automatic fallback)
• curl for local testing

Scaffold the Project

Create a fresh Worker using the standard JavaScript template:

wrangler init llm-proxy
cd llm-proxy

When prompted, choose “Hello World” (JavaScript). Wrangler creates src/index.js and wrangler.toml.

Configure Wrangler and Secrets

Point the Worker at your upstream base URL. Edit wrangler.toml:

name = "llm-proxy"
main = "src/index.js"
compatibility_date = "2024-09-01"

[vars]
UPSTREAM_BASE = "https://api.openai.com/v1"

Store the API key as a secret, not a var:

wrangler secret put API_KEY
# paste your key when prompted

Write the Proxy Logic

Replace src/index.js with a module that forwards the request path and body, injects auth, and returns the upstream response stream:

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const upstream = new URL(env.UPSTREAM_BASE);
    upstream.pathname = url.pathname;
    upstream.search = url.search;

    const headers = new Headers(request.headers);
    headers.delete('host');
    headers.set('authorization', `Bearer ${env.API_KEY}`);

    const upstreamReq = new Request(upstream, {
      method: request.method,
      headers,
      body: request.body,
      redirect: 'follow'
    });

    const response = await fetch(upstreamReq);
    const respHeaders = new Headers(response.headers);
    respHeaders.delete('transfer-encoding');

    return new Response(response.body, {
      status: response.status,
      headers: respHeaders
    });
  }
};

This cloudflare worker proxy openai-compatible implementation passes through streaming responses because response.body is a ReadableStream.

Run Locally and Verify

Start the dev server:

wrangler dev

In another shell, send a chat completion:

curl -X POST http://localhost:8787/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Say hi"}]}'

Expected Output

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "Hi there!" }, "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 5, "completion_tokens": 3, "total_tokens": 8 }
}

Streaming and Headers

OpenAI-compatible APIs support Server-Sent Events when "stream": true. The code above already handles it: the fetch response body is streamed straight to the client. Test with:

curl -X POST http://localhost:8787/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Count 1 to 3"}],"stream":true}'

You should see incremental data: {...} chunks. No extra code is needed because Workers natively proxy streams.

One nuance: upstreams sometimes send set-cookie or strict CORS headers. Strip anything you don’t want echoed:

respHeaders.delete('set-cookie');

Deploy to Edge

Publish the Worker:

wrangler deploy

Your cloudflare worker proxy openai-compatible is now live at https://llm-proxy.<subdomain>.workers.dev. Point your OpenAI SDK at that URL by setting baseURL.

from openai import OpenAI
client = OpenAI(base_url="https://llm-proxy.<subdomain>.workers.dev", api_key="dummy")
print(client.chat.completions.create(model="gpt-3.5-turbo", messages=[{"role":"user","content":"hi"}]).choices[0].message.content)

Adding CORS for Browser Apps

If you call the proxy from a browser, handle preflight:

async fetch(request, env) {
  if (request.method === 'OPTIONS') {
    return new Response(null, {
      headers: {
        'access-control-allow-origin': '*',
        'access-control-allow-headers': 'content-type, authorization',
        'access-control-allow-methods': 'POST, GET, OPTIONS'
      }
    });
  }
  // ... existing proxy logic, but add CORS header to respHeaders:
  respHeaders.set('access-control-allow-origin', '*');
  // ...
}

Forwarding Routing Directives (Optional)

Some gateways, including n4n.ai, honor client routing directives and forward provider cache-control hints. Pass them through explicitly:

if (request.headers.get('x-model-router')) {
  headers.set('x-model-router', request.headers.get('x-model-router'));
}
if (request.headers.get('cache-control')) {
  headers.set('cache-control', request.headers.get('cache-control'));
}

This lets clients pin a provider or leverage prompt caching without the Worker interpreting the semantics.

Operational Notes

• Worker CPU time limits apply; streaming avoids buffering but long generations can hit wall-clock limits on free tiers.
• Use wrangler tail to debug live requests.
• Store keys via wrangler secret, never commit them to wrangler.toml.
• The pattern scales to any OpenAI-compatible endpoint—swap UPSTREAM_BASE to target a different vendor or gateway.