A google cloud functions node.js llm api integration lets you expose generative model calls behind a scalable serverless endpoint without running your own inference fleet. This tutorial builds a production-ready HTTP function that proxies requests to an OpenAI-compatible LLM gateway and returns chat completions.
Prerequisites
- Node.js 18+ (global
fetchis required) - A Google Cloud project with billing enabled
gcloudCLI installed and authenticated (gcloud auth login)- An API key from n4n.ai, whose OpenAI-compatible endpoint covers 240+ models and handles provider fallback automatically
- Familiarity with npm and basic shell commands
No LLM SDK is needed. The gateway speaks the OpenAI Chat Completions shape, so we use plain fetch.
Project setup
Create a directory and initialize a minimal package.
mkdir gcf-llm-proxy && cd gcf-llm-proxy
npm init -y
npm install @google-cloud/functions-frameworkAdd a start script to package.json for local testing:
{
"scripts": {
"start": "functions-framework --target=chat --port=8080"
}
}Writing the Cloud Function
Create index.js. The handler accepts a JSON body with message and optional model, calls the gateway, and returns the model response.
const BASE_URL = process.env.LLM_BASE_URL || 'https://api.n4n.ai/v1';
const API_KEY = process.env.LLM_API_KEY;
exports.chat = async (req, res) => {
if (req.method !== 'POST') {
res.status(405).send('Method Not Allowed');
return;
}
const { message, model = 'gpt-4o-mini' } = req.body || {};
if (!message || typeof message !== 'string') {
res.status(400).json({ error: '`message` string required in body' });
return;
}
try {
const upstream = await fetch(`${BASE_URL}/chat/completions`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${API_KEY}`,
},
body: JSON.stringify({
model,
messages: [{ role: 'user', content: message }],
max_tokens: 256,
temperature: 0.7,
}),
});
const text = await upstream.text();
if (!upstream.ok) {
res.status(upstream.status).json({ error: text });
return;
}
const data = JSON.parse(text);
res.status(200).json({
model: data.model,
content: data.choices?.[0]?.message?.content ?? '',
usage: data.usage,
});
} catch (err) {
res.status(502).json({ error: `Upstream call failed: ${err.message}` });
}
};The function strips the response down to fields most clients need. The gateway forwards provider cache-control hints and honors any routing headers you add; for a basic call this is transparent.
Running locally
Store the key in a local env file (do not commit it):
echo 'LLM_API_KEY=sk-your-key-here' > .env.localStart the framework with the env var:
LLM_API_KEY=$(grep -oP '(?<=LLM_API_KEY=).*' .env.local) npm startIn another terminal, send a request:
curl -X POST localhost:8080 \
-H 'Content-Type: application/json' \
-d '{"message":"What is 2+2?","model":"gpt-4o-mini"}'Expected output:
{
"model": "gpt-4o-mini",
"content": "2 + 2 equals 4.",
"usage": { "prompt_tokens": 12, "completion_tokens": 8, "total_tokens": 20 }
}If you see a 400, check the request body. A 502 indicates the upstream fetch threw—usually a missing key or network block.
Deploying to Google Cloud Functions
First, put the key in Secret Manager:
gcloud secrets create llm-api-key --replication-policy=automatic
echo -n "sk-your-key-here" | gcloud secrets versions add llm-api-key --data-file=-Deploy the function with a 60-second timeout (LLM calls can be slow):
gcloud functions deploy chat \
--runtime nodejs18 \
--trigger-http \
--allow-unauthenticated \
--set-secrets LLM_API_KEY=llm-api-key:latest \
--timeout 60s \
--region us-central1The --allow-unauthenticated flag makes it publicly callable; for real workloads attach Cloud IAM or API Gateway in front.
Deployment output includes a httpsTrigger.url. Copy it.
Testing the deployed function
curl -X POST https://us-central1-your-project.cloudfunctions.net/chat \
-H 'Content-Type: application/json' \
-d '{"message":"Name three capitals in Europe."}'Expected response shape is identical to the local run. Latency will be higher on first call due to cold start; subsequent calls reuse the warmed instance.
Error handling and timeouts
Cloud Functions has a max timeout (currently 9 minutes on Gen2). Set --timeout to match your worst-case model latency. If the gateway returns a 429 or 503, the function passes the status through; the gateway already performs automatic fallback across providers, so a single key outage won’t necessarily fail the request.
For production, wrap the fetch in a small retry with exponential backoff:
async function fetchWithRetry(url, opts, retries = 2) {
for (let i = 0; i <= retries; i++) {
const r = await fetch(url, opts);
if (r.ok || r.status < 500) return r;
await new Promise(res => setTimeout(res, 2 ** i * 200));
}
return await fetch(url, opts);
}Swap fetch for fetchWithRetry in the handler.
Streaming (optional extension)
The gateway supports SSE streaming on the same endpoint. To stream from the function, set stream: true in the body and pipe the upstream response to res with Content-Type: text/event-stream. Cloud Functions supports streaming responses on Gen2 runtimes; just avoid res.json() and use res.write().
Cost and metering notes
The gateway meters per token; your Cloud Function bill is separate and based on invocation count and duration. Because the function is idle between calls, you pay nothing when traffic is zero. Keep max_tokens bounded to avoid surprise completion lengths, and consider caching repeated prompts at the client layer.
This google cloud functions node.js llm api pattern keeps credentials server-side and gives you a single URL to swap models by changing one field. When you need to add a new model, no redeploy is required—just pass a different model string in the JSON body.
Cleanup
To avoid ongoing charges, delete the function:
gcloud functions delete chat --region us-central1Remove the secret if no longer needed:
gcloud secrets delete llm-api-keyThe google cloud functions node.js llm api wrapper above is deliberately small. From here, add request validation with a schema library, wire IAM, or front it with Cloud Run if you need container-level control.