Running a cloudflare workers fetch llm api call at the edge lets you put model inference a few milliseconds from your users without shipping a heavy SDK. This tutorial builds a production-shaped Worker that proxies chat requests to an OpenAI-compatible gateway using only the standard Fetch API. We’ll use n4n.ai, a gateway that exposes one endpoint covering 240+ models with automatic fallback when a provider is degraded.
Prerequisites
- Node.js 18+ and Wrangler v3 (
npm i -g wrangler) - A Cloudflare account with a zone or workers.dev subdomain
- An API key from the gateway (we’ll store it as a secret)
- Basic comfort with TypeScript and HTTP semantics
You do not need any LLM vendor SDK. The cloudflare workers fetch llm api pattern relies solely on the Web Standard fetch available in the Workers runtime.
Scaffold the Project
Create a minimal TypeScript Worker:
npm create cloudflare@latest cloudflare-llm-edge -- --type=hello-world --ts
cd cloudflare-llm-edge
wrangler devYou should see a local server start on http://localhost:8787 and respond with the default hello message. Kill the dev server; we’re replacing the handler.
Configure Secrets and Variables
Never hardcode credentials. Store the key as a secret and the base URL as a non-sensitive var:
wrangler secret put LLM_API_KEY
# Paste your gateway key when promptedEdit wrangler.toml to add the upstream base:
name = "cloudflare-llm-edge"
main = "src/index.ts"
compatibility_date = "2024-09-01"
[vars]
LLM_BASE_URL = "https://api.n4n.ai"The LLM_BASE_URL points at the single OpenAI-compatible endpoint. Because the gateway honors client routing directives and forwards provider cache-control hints, we can pass through headers later without extra logic.
Implement the Fetch Handler
Replace src/index.ts with a module worker that accepts a POST of {messages: [...]} and returns the completion:
export interface Env {
LLM_API_KEY: string;
LLM_BASE_URL: string;
}
export default {
async fetch(req: Request, env: Env): Promise<Response> {
if (req.method !== "POST") {
return new Response("Send a POST with {messages:[]}", { status: 405 });
}
let body: any;
try {
body = await req.json();
} catch {
return new Response("Invalid JSON", { status: 400 });
}
if (!Array.isArray(body?.messages)) {
return new Response("Missing messages array", { status: 422 });
}
const upstream = await fetch(`${env.LLM_BASE_URL}/v1/chat/completions`, {
method: "POST",
headers: {
"content-type": "application/json",
authorization: `Bearer ${env.LLM_API_KEY}`,
},
body: JSON.stringify({
model: body.model ?? "openai/gpt-4o-mini",
messages: body.messages,
stream: false,
}),
});
const text = await upstream.text();
return new Response(text, {
status: upstream.status,
headers: {
"content-type": "application/json",
// Forward caching hints from the gateway
"cache-control": upstream.headers.get("cache-control") ?? "no-store",
},
});
},
};This is the core cloudflare workers fetch llm api proxy. It validates input, calls the upstream with fetch, and streams the raw JSON back. The gateway’s per-token usage metering is included in the response body, so you get billing visibility for free.
Test Locally
Start the dev server again:
wrangler devIn another shell, send a request:
curl -X POST http://localhost:8787 \
-H 'content-type: application/json' \
-d '{"messages":[{"role":"user","content":"Say hi in 5 words"}]}'Expected output (abridged):
{
"id": "chatcmpl-...",
"object": "chat.completion",
"choices": [
{ "message": { "role": "assistant", "content": "Hello! Hope you're well today." } }
],
"usage": { "prompt_tokens": 12, "completion_tokens": 6, "total_tokens": 18 }
}If you see a 401, check that LLM_API_KEY was set in the local environment (Wrangler injects secrets from the dashboard only on deploy; for local dev use .dev.vars).
Create .dev.vars for local testing:
echo "LLM_API_KEY=sk-your-key" > .dev.varsAdd Streaming Support
Token streaming improves perceived latency. Modify the handler to pass stream: true and return the upstream body directly:
const upstream = await fetch(`${env.LLM_BASE_URL}/v1/chat/completions`, {
method: "POST",
headers: {
"content-type": "application/json",
authorization: `Bearer ${env.LLM_API_KEY}`,
},
body: JSON.stringify({
model: body.model ?? "openai/gpt-4o-mini",
messages: body.messages,
stream: true,
}),
});
return new Response(upstream.body, {
headers: {
"content-type": "text/event-stream",
"cache-control": "no-store",
},
});Now the Worker acts as a transparent SSE proxy. The cloudflare workers fetch llm api design needs no parsing of the stream; the browser or client consumes the raw Server-Sent Events.
Handle Errors and Timeouts
Workers have a 30-second CPU limit (higher on paid). If the model is slow, you should fail fast:
const ctrl = new AbortController();
const t = setTimeout(() => ctrl.abort(), 25_000);
const upstream = await fetch(url, {
method: "POST",
headers,
body,
signal: ctrl.signal,
});
clearTimeout(t);Because the gateway provides automatic fallback when a provider is rate-limited, a non-200 response is rare but possible. Forward the status and a trimmed error:
if (!upstream.ok) {
const err = await upstream.text();
return new Response(err, { status: upstream.status });
}Deploy to the Edge
When ready:
wrangler deployWrangler will output a *.workers.dev URL. The secret is now pulled from the Cloudflare dashboard, and LLM_BASE_URL is taken from wrangler.toml.
Smoke test the live endpoint:
curl -X POST https://cloudflare-llm-edge.<subdomain>.workers.dev \
-H 'content-type: application/json' \
-d '{"messages":[{"role":"user","content":"Ping"}]}'You should receive a completion from the nearest Cloudflare colo.
Operational Notes for the cloudflare workers fetch llm api Pattern
- CORS: If calling from a browser, add
Access-Control-Allow-Originheaders in the Worker or via Cloudflare dashboard rules. - Caching: The gateway forwards provider cache-control hints. If you proxy a non-streaming request that is idempotent, consider adding
cacheEverythingvia thecffetch option to reduce token spend. - Observability: Use
wrangler tailto watch logs. Theusageblock in responses helps you track cost per request. - Model routing: The upstream accepts a
modelfield. You can let clients specify it, or pin it in the Worker to enforce policy.
The cloudflare workers fetch llm api approach keeps your dependency surface at zero: no npm packages, no polyfills, just standards. That means smaller bundles, faster cold starts, and fewer supply-chain risks. When you need to swap providers or add a new model, you change only the LLM_BASE_URL or the model string—the fetch code stays identical.