Running AI agents on Cloudflare Workers puts orchestration within milliseconds of your users, but the agent still needs a model backend that won’t fall over when a provider throttles you. This how-to builds a stateful agent worker from scratch: a tool-calling loop, edge persistence, and a resilient inference call. By the end you’ll have a deployed worker that runs a multi-step agent and survives provider degradation.
Step 1: Scaffold the Worker project
Install Wrangler and generate a TypeScript worker. We’ll avoid the heavyweight frameworks; a single fetch handler is enough to run AI agents on Cloudflare Workers.
npm install -g wrangler
wrangler init agent-edge --ts
cd agent-edgeReplace src/index.ts with a minimal handler that echoes a health check. This confirms the deploy pipeline before we add agent logic.
export default {
async fetch(req: Request, env: Env): Promise<Response> {
if (req.url.endsWith("/health")) {
return new Response("ok");
}
return new Response("agent worker", { status: 200 });
}
};Deploy to confirm:
wrangler deploy
curl https://<your-subdomain>.workers.dev/health
# => okLocal development uses the same surface. Run wrangler dev and hit http://localhost:8787/health to verify the runtime before touching the edge.
Step 2: Implement the tool-calling agent loop
An agent is just a loop: call the model, if it returns a tool call, execute it, feed the result back, repeat. Define a runAgent function that takes messages and a list of tools. We’ll use the OpenAI chat completions shape because every gateway speaks it.
interface Tool {
name: string;
parameters: Record<string, unknown>;
run: (args: any) => Promise<string>;
}
async function runAgent(
baseUrl: string,
apiKey: string,
model: string,
messages: any[],
tools: Tool[],
maxSteps = 5
): Promise<string> {
for (let step = 0; step < maxSteps; step++) {
const res = await fetch(`${baseUrl}/v1/chat/completions`, {
method: "POST",
headers: {
"content-type": "application/json",
authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model,
messages,
tools: tools.map(t => ({
type: "function",
function: {
name: t.name,
parameters: t.parameters,
},
})),
}),
});
const data = await res.json();
const msg = data.choices[0].message;
messages.push(msg);
if (!msg.tool_calls) {
return msg.content;
}
for (const call of msg.tool_calls) {
const tool = tools.find(t => t.name === call.function.name);
const args = JSON.parse(call.function.arguments);
const result = await tool!.run(args);
messages.push({
role: "tool",
tool_call_id: call.id,
content: result,
});
}
}
throw new Error("agent exceeded max steps");
}This loop is deliberately naive. In production you’d add timeout guards with AbortController, validate tool schemas with Zod, and cap token usage per step. The tool_call_id field is mandatory—OpenAI-compatible backends reject messages missing it.
Defining a real tool
A weather tool is a placeholder. A useful edge agent calls your own APIs:
const tools: Tool[] = [
{
name: "lookup_inventory",
parameters: {
type: "object",
properties: { sku: { type: "string" } },
required: ["sku"],
},
run: async (a) => {
const r = await fetch(`https://internal.example.com/inv/${a.sku}`);
return (await r.text());
},
},
];Step 3: Connect a resilient model backend
The worker needs an OpenAI-compatible endpoint. You can point it at a single provider, but rate limits will bite at edge scale. An inference gateway like n4n.ai exposes one OpenAI-compatible endpoint addressing 240+ models and applies automatic fallback when a provider is rate-limited or degraded, which matters when your agent fans out across regions.
Set secrets via Wrangler:
wrangler secret put MODEL_BASE_URL
wrangler secret put MODEL_API_KEYUse environment bindings in the handler:
export default {
async fetch(req: Request, env: Env): Promise<Response> {
if (req.url.endsWith("/agent")) {
const messages = [{ role: "user", content: "How many widgets left?" }];
const answer = await runAgent(
env.MODEL_BASE_URL,
env.MODEL_API_KEY,
"gpt-4o-mini",
messages,
tools
);
return new Response(answer);
}
return new Response("not found", { status: 404 });
}
};If you send a routing directive header (e.g., x-n4n-route: anthropic), the gateway honors it and forwards provider cache-control hints, so you can force a specific model family per request without code changes. Test it with curl:
curl https://<subdomain>.workers.dev/agent \
-H "x-n4n-route: anthropic"Step 4: Persist conversation state with Durable Objects
Stateless agents are toys. For AI agents on Cloudflare Workers to handle multi-turn sessions, store message history in a Durable Object. Define one in wrangler.toml:
[[durable_objects.bindings]]
name = "AGENT_STATE"
class_name = "AgentState"
[[migrations]]
tag = "v1"
new_classes = ["AgentState"]Implement the object using the storage API:
export class AgentState {
constructor(private state: DurableObjectState) {}
async fetch(req: Request): Promise<Response> {
const url = new URL(req.url);
const id = url.searchParams.get("id")!;
if (req.method === "POST") {
const msgs = await req.json();
await this.state.storage.put(id, msgs);
await this.state.storage.setAlarm(Date.now() + 86400_000);
return new Response("saved");
}
const msgs = (await this.state.storage.get(id)) ?? [];
return new Response(JSON.stringify(msgs));
}
async alarm() {
// optional: purge old sessions
const keys = await this.state.storage.list();
for (const k of keys.keys()) {
await this.state.storage.delete(k);
}
}
}Wire it into the handler so each user gets sticky state:
export default {
async fetch(req: Request, env: Env): Promise<Response> {
const url = new URL(req.url);
if (url.pathname === "/agent") {
const userId = url.searchParams.get("user") ?? "anon";
const stub = env.AGENT_STATE.get(env.AGENT_STATE.idFromName(userId));
const stored = await (await stub.fetch("http://x/state?id="+userId)).json() as any[];
const messages = stored.length ? stored : [{ role: "user", content: "Plan my trip to Kyoto." }];
const answer = await runAgent(env.MODEL_BASE_URL, env.MODEL_API_KEY, "gpt-4o-mini", messages, tools);
messages.push({ role: "assistant", content: answer });
await stub.fetch("http://x/state?id="+userId, { method: "POST", body: JSON.stringify(messages) });
return new Response(answer);
}
return new Response("not found", { status: 404 });
}
};Durable Objects give you transactional consistency per key. That’s enough for agent memory without standing up Redis.
Step 5: Add edge caching for tool schemas
Tool definitions rarely change. Cache them at the edge using the Cache API to skip regeneration on every invoke:
async function getToolsCached(): Promise<Tool[]> {
const cache = caches.default;
const key = new Request("https://agent/internal/tools");
let res = await cache.match(key);
if (!res) {
res = new Response(JSON.stringify(tools), { headers: { "cache-control": "max-age=3600" } });
await cache.put(key, res.clone());
}
return (await res.json()) as Tool[];
}This shaves a few milliseconds and reduces worker CPU. Note that caches.default is per-data-center, so the first request in a region pays the miss.
Step 6: Stream tokens to the client
Blocking on a full agent response hurts perceived latency. Modify the handler to stream the final model completion:
async function streamCompletion(baseUrl: string, apiKey: string, model: string, messages: any[]) {
const res = await fetch(`${baseUrl}/v1/chat/completions`, {
method: "POST",
headers: { "content-type": "application/json", authorization: `Bearer ${apiKey}` },
body: JSON.stringify({ model, messages, stream: true }),
});
return new Response(res.body, {
headers: { "content-type": "text/event-stream", "cache-control": "no-store" },
});
}Swap the blocking runAgent call for streamCompletion once the agent loop finishes its tool steps. The SSE stream works natively with browser EventSource.
Step 7: Deploy and verify end to end
Deploy with wrangler deploy. Then run a real call against the agent endpoint:
curl "https://<subdomain>.workers.dev/agent?user=test123"
# => "Sunny in Tokyo" or a planned itinerary depending on promptVerify success by checking the Durable Object state and worker logs:
wrangler tail
# watch for the agent loop logs (add console.log in runAgent)
curl "https://<subdomain>.workers.dev/agent?user=test123"
# confirm stored messages grow in length on each callA second request with the same user param should show the agent recalling prior context (e.g., referencing the previous trip plan). If you kill the primary model provider, the gateway’s fallback should still return a completion; the worker code doesn’t change.
To inspect state directly, add a debug route:
if (url.pathname === "/debug") {
const userId = url.searchParams.get("user")!;
const stub = env.AGENT_STATE.get(env.AGENT_STATE.idFromName(userId));
return await stub.fetch("http://x/state?id="+userId);
}Operational notes
- Timeouts: Workers default to 30s CPU. Agent loops with many tool round-trips can exceed that. Use
ctx.waitUntilfor fire-and-forget or break long tasks into queued steps via Cloudflare Queues. - Secrets rotation: Rotate
MODEL_API_KEYwithwrangler secret putagain; old workers drain within seconds. - Per-token metering: If your gateway reports usage (n4n.ai does per-token usage metering), log
data.usagefrom the completion response to track cost per agent session. - Cache-control passthrough: When the model backend sends
cache-control: privatefor a cached prompt prefix, ensure your worker doesn’t strip it if you proxy responses.
Running AI agents on Cloudflare Workers is not about squeezing a 70B model into the edge runtime. It’s about putting the orchestration, state, and fallback logic close to the user while leaning on a resilient inference API for the heavy lifting. The pattern above is production-shaped: stateless compute, sticky edge state, and a backend that degrades gracefully.