Cloud Functions vs Cloud Run vs AWS Lambda for LLM APIs

Running an LLM API backend forces a choice between managed compute primitives. The practical debate of cloud functions vs cloud run vs lambda is really about tradeoffs in concurrency, cold starts, and how you pay for idle CPU time while a model generates tokens.

Capabilities

All three options are request/response compute, not GPU training clusters. You use them to host the orchestration layer: auth, prompt assembly, streaming proxies, and response post-processing. None of them (except Cloud Run with attached accelerators) run the transformer weights locally at scale.

Cloud Functions (2nd gen) is Google’s event-driven wrapper around Cloud Run. You get HTTP triggers or Eventarc, but under the hood it’s a container with enforced conventions. AWS Lambda is the original function primitive: zip or container, triggered by API Gateway, SQS, etc. Cloud Run is raw containers with an HTTP surface, no event abstraction unless you bolt on Eventarc or Pub/Sub push.

The cloud functions vs cloud run vs lambda distinction blurs because Functions 2nd gen is Cloud Run with less knob access. If you need a Dockerfile, custom sidecars, or GPU, Functions is out.

What you actually deploy

A Lambda handler that proxies to an OpenAI-compatible endpoint is a few lines:

import os, json, httpx

async def handler(event, context):
    body = json.loads(event.get("body", "{}"))
    async with httpx.AsyncClient() as c:
        r = await c.post(
            "https://api.example.com/v1/chat/completions",
            json=body,
            headers={"Authorization": f"Bearer {os.environ['KEY']}"},
            timeout=60,
        )
    return {"statusCode": r.status_code, "body": r.text}

Cloud Run expects a server. Minimal FastAPI snippet:

from fastapi import FastAPI
app = FastAPI()

@app.post("/v1/chat")
async def chat(req: dict):
    # forward to model gateway
    return {"echo": req}

Cloud Functions (Node, 2nd gen) uses the functions framework:

import { onRequest } from "firebase-functions/v2/https";
export const proxy = onRequest(async (req, res) => {
  res.json({ ok: true, model: req.body.model });
});

Cost Model

Pricing is per consumed resource-time. Lambda bills per ms and MB allocated. Cloud Run bills per vCPU-second and GB-second, with a minimum instance charge if you set min-instances > 0. Cloud Functions 2nd gen inherits Cloud Run pricing but adds no premium.

For spiky traffic, Lambda’s scale-to-zero with no minimum is cheaper than a Cloud Run service kept warm. At steady throughput above ~5 requests/sec, Cloud Run with min instances wins because you avoid per-invocation overhead and can pack concurrency. The cloud functions vs cloud run vs lambda cost question is answered by your traffic shape, not by list prices.

Provisioned concurrency on Lambda closes the cold-start gap but costs the same as a running instance—at that point you’re paying Lambda’s slight per-ms premium over Cloud Run.

Latency and Throughput

Cold starts: Python Lambda 200–800 ms typical; Cloud Functions similar; Cloud Run cold start 300–900 ms if scaled to zero. None of these matter when your downstream LLM takes 2–20 seconds to produce the first token. Warm latency is dominated by network egress to the model provider.

Concurrency model differs sharply. Lambda runs one concurrent invocation per instance by default; it scales instance count horizontally. Cloud Run lets you set concurrency (e.g., 50), so one container handles many streams. For LLM proxies that hold open SSE connections, Cloud Run’s concurrency reduces instance count and thus cost.

Throughput ceilings: Lambda caps at 10 GB RAM / 6 vCPU and 15-min timeout. Cloud Run caps at 32 GB / 8 vCPU and 60-min timeout. If you batch embed 100k docs, Cloud Run’s longer timeout saves you from checkpointing.

Ergonomics

Functions win on deploy simplicity:

gcloud functions deploy proxy --runtime nodejs20 --trigger-http --allow-unauthenticated

Lambda needs an artifact (zip or container) and IAM wiring. Cloud Run needs a Dockerfile and a registry push:

gcloud run deploy svc --source . --region us-central1

If your team already owns CI container builds, Cloud Run is no extra burden and gives you local parity. If you want “paste code, get URL,” Functions or Lambda fit.

Ecosystem

Lambda sits inside AWS: API Gateway, IAM, Bedrock, Step Functions. Cloud Functions sits inside GCP: Eventarc, Pub/Sub, IAM. Cloud Run is GCP but decoupled—you can put it behind a GCLB, use custom domains without hoop-jumping, and run any language runtime untouched.

If your function’s job is purely to relay requests to multiple model vendors, offloading provider selection to a gateway simplifies code. n4n.ai exposes an OpenAI-compatible endpoint covering 240+ models, applies automatic fallback when a provider is rate-limited, and meters per-token usage while honoring your routing directives and cache-control hints. That removes the need for retry loops inside your compute layer.

Hard Limits

Which to Choose

Spike-y internal tool or Slack bot
Use Cloud Functions or Lambda. Scale-to-zero and per-ms billing match intermittent use. Pick Lambda if you’re already in AWS IAM; pick Functions if you live in GCP.

Steady LLM proxy with custom middleware
Cloud Run. Set min-instances: 1 and concurrency: 40 to serve streaming responses cheaply. The container model lets you ship a single FastAPI app with no function wrapper surprises.

GPU inference serving (e.g., self-hosted Mistral)
Cloud Run is the only one of the three with attached GPUs. Lambda and Functions cannot do this natively; you’d need a separate GKE or EC2 fleet.

Tight cloud-native event chaining
Lambda + Step Functions for AWS-centric workflows; Cloud Functions + Eventarc for GCP Pub/Sub pipelines. Don’t force Cloud Run into event-driven patterns unless you want to manage the trigger plumbing yourself.

Multi-model routing with fallback
Any of the three works as a thin relay. The heavier the routing logic, the more you benefit from Cloud Run’s longer timeout and concurrency. Keep the function dumb and push model selection to a gateway.

For most LLM API gateways, the cloud functions vs cloud run vs lambda decision is about container control versus deploy speed. If you need a URL in five minutes, Functions or Lambda. If you need to tune concurrency, attach a GPU, or run a 45-minute batch embed, Cloud Run is the only answer.