Async LLM tasks: webhooks, polling, or server-sent events

Long-running LLM jobs—batch embeddings, multi-document summarization, agent loops—don’t fit a synchronous request/response model. The choice between webhooks vs polling vs sse llm determines how your service learns that a job finished, how much load you put on the network, and how quickly you can act on results. Get it wrong and you’ll either burn compute on empty polls or lose results when a callback drops.

Patterns at a glance

Webhooks

The provider POSTs a JSON payload to a URL you expose when the job completes. You need a public endpoint, idempotency handling, and signature verification.

@app.post("/llm-callback")
async def llm_callback(req: Request):
    sig = req.headers.get("X-Signature")
    if not verify(sig, await req.body()):
        raise HTTPException(401)
    job = await req.json()
    await process_result(job["id"], job["output"])
    return {"ok": True}

Polling

You submit the job, get a job ID, then repeatedly GET its status until state flips to completed.

job_id = submit_llm_job(payload)
while True:
    status = get_job(job_id)
    if status["state"] == "completed":
        break
    time.sleep(2)
result = status["result"]

Server-Sent Events

A single HTTP connection stays open; the server streams incremental tokens or a final done event.

const es = new EventSource("/v1/stream?job_id=123");
es.onmessage = (e) => {
  const data = JSON.parse(e.data);
  if (data.event === "token") updateUI(data.text);
  if (data.event === "done") es.close();
};

Head-to-head comparison

Capabilities

Webhooks push a completion event to you. They suit jobs that run minutes or hours; the provider doesn’t keep a connection open. You receive the full payload exactly once (with retries). Polling only tells you status when you ask. It can’t deliver partial output—you get the whole result at the end, or you must poll a separate streaming endpoint. SSE bridges the gap: it streams tokens as they generate, then sends a terminal event. For LLM tasks where users watch text appear, SSE is the only pattern that gives progressive rendering without custom websockets.

The webhooks vs polling vs sse llm decision often starts here: do you need intermediate output or just the final answer? If you’re generating a 10k-token report, SSE lets the UI paint paragraphs live. If you’re classifying 1M rows nightly, webhooks let you batch-insert when each finishes.

Webhooks also support multi-event callbacks: started, progress, completed, failed. Polling can simulate progress by exposing a percent field, but that still costs a request each time. SSE can emit custom events with metadata, but the connection must survive the job lifetime.

Cost model

Polling hides a tax: every empty GET /job/123 costs a request. At 2-second intervals for a 5-minute job, that’s 150 calls. If your gateway meters per request and per token, those polls are pure overhead. Webhooks incur cost only when the job finishes—one POST in, one POST out. SSE holds a connection; the server pays memory and file descriptors per client, but you avoid request storms.

Do the math for a system with 10k jobs/day averaging 3 minutes each, polled every 5 seconds: 36 polls per job × 10k = 360k status calls daily. At $0.0001 per call that’s $36/day just for polling. Webhooks would be 10k callbacks. (Numbers are illustrative; actual rates vary.)

If you run behind n4n.ai’s OpenAI-compatible endpoint, per-token usage metering means you pay for generation regardless of delivery pattern, but the request overhead is separate. Webhooks keep that overhead minimal.

SSE cost is indirect: a Python asyncio server handles ~10k concurrent connections per GB RAM; beyond that you scale horizontally. That’s fine for consumer apps, painful for IoT fleets.

Latency and throughput

Webhook latency is the network round-trip after completion—usually sub-second. Polling latency is your interval plus jitter; set it too short and you hit rate limits, set it too long and users wait. SSE delivers the first token in milliseconds after generation starts, but total completion latency is identical to other patterns.

Throughput: webhooks scale to millions of jobs because the provider queues deliveries. Polling throughput is capped by how many status calls your client can make (typically a few thousand per minute). SSE throughput is capped by concurrent sockets—browsers limit ~6 per domain, servers limit by RAM and epoll slots.

The webhooks vs polling vs sse llm spectrum also affects tail latency. A webhook that retries with exponential backoff may deliver late if your endpoint 500s. Polling never misses but may lag. SSE fails silently if the proxy kills the connection; you need a resume token.

Ergonomics

Polling wins on simplicity. A cron job or loop with sleep works behind any NAT. Webhooks force you to stand up a secure endpoint, validate signatures, and handle retries with backoff. Local testing needs a tunnel (ngrok) or a mock receiver. SSE is easy in browsers via EventSource, but server-side you must manage reconnection and heartbeat comments to keep proxies happy.

# SSE heartbeat to avoid proxy timeout
async def stream_job(job_id):
    while not done:
        yield ": ping\n\n"
        await asyncio.sleep(15)

Webhooks require a dead-letter queue. If the provider gives up after 3 tries, you must reconcile by listing incomplete jobs. Polling gives you natural reconciliation: just query state=pending. SSE gives neither; if the client disconnects, the server may discard the stream unless it persists to a log.

Ecosystem

Webhooks are the lingua franca of async SaaS—Stripe, GitHub, Twilio. Every language has a web framework to receive them. Polling is just HTTP, so it’s universal. SSE is an HTML5 standard with solid browser support and libraries for Node, Python, Go. It loses to WebSockets for bidirectional needs, but for LLM output it’s enough.

OpenAPI documents webhooks under callbacks; polling is just get operations; SSE uses text/event-stream media type. Tooling like Postman can mock webhooks, curl can poll, and curl -N can read SSE.

Limits

Webhook providers often cap retry attempts (e.g., 3 tries) and then dead-letter the job; you must build a reconciliation loop. Polling hits rate limits on the status endpoint—typically 100–1000 req/min. SSE connections die at proxy idle timeouts (30–60s) unless you send comments. Also, some serverless platforms freeze connections after the response ends, making SSE impossible without a custom runtime.

Webhooks also require a stable public IP or domain; if you rotate URLs, you must re-register. Polling can be blocked by egress firewalls but works in locked-down VPCs. SSE is blocked by older corporate proxies that buffer responses.

Which to choose

Batch processing, nightly embeddings, offline summarization: Use webhooks. You get pushed when done, no wasted polls, and can queue results in a durable queue. Add a weekly scan for orphaned jobs.

Internal scripts, quick prototypes, jobs behind a firewall with no public IP: Use polling. It’s the path of least resistance; just respect rate limits and use exponential backoff. Cap concurrency to avoid self-DoS.

Chatbots, live document drafting, any UI showing tokens stream: Use SSE. It’s the only pattern that renders generation progressively without standing up a websocket server. Send heartbeats and store the last event ID for resume.

High-volume mixed workloads: Combine. Submit via API, stream via SSE if the client is online, and register a webhook as fallback for disconnected clients. The webhooks vs polling vs sse llm trade-off isn’t exclusive—gateways that honor routing directives can fan out as needed.

If you must pick one for a general LLM platform, SSE for interactive and webhooks for background. Polling is the fallback when network constraints block the other two.