Debugging env variable issues in serverless LLM deploys

Env variable errors serverless llm deploy usually don’t show up until your function cold-starts in production and throws a cryptic “undefined API key” deep inside a model client. The local dev server read your .env file, but the deployed artifact has no such luxury. Below is the debugging path we use to turn those opaque 500s into a five-minute fix.

Step 1: Reproduce the exact runtime environment locally

Serverless platforms inject env vars at runtime, not from a .env file you ship. Pull the real values into a local harness before blaming the provider.

For AWS Lambda, export the live configuration:

aws lambda get-function-configuration \
  --function-name my-llm-proxy \
  --query 'Environment.Variables' > live_env.json

Then invoke your handler locally with those vars applied:

jq -r 'to_entries[] | "\(.key)=\(.value)"' live_env.json > .env.live
env $(cat .env.live) node -e "require('./dist/handler').handler({})"

If you run containers, do the same with docker run --env-file .env.live. The goal is to prove the code works when the vars are present. Env variable errors serverless llm deploy often die right here because the local .env was never uploaded.

For Cloudflare Workers, wrangler dev reads *.vars from wrangler.toml, but production uses secret bindings. Simulate both:

wrangler dev --env dev --var LLM_BASE_URL:https://example.com

Step 2: Inventory every variable your LLM code reads

Grep the codebase for process.env (or os.environ in Python) and list what’s mandatory versus optional.

grep -rn "process.env" src/ | grep -v "NODE_ENV" | sort

A typical LLM deploy touches at least three: a provider key, a base URL, and a model name. Mark each as required in a single config module:

// config.js
export const config = {
  llmApiKey: process.env.LLM_API_KEY,
  llmBaseUrl: process.env.LLM_BASE_URL ?? 'https://api.openai.com/v1',
  llmModel: process.env.LLM_MODEL ?? 'gpt-4o-mini',
};

In Python:

import os

CONFIG = {
    "api_key": os.environ.get("LLM_API_KEY"),
    "base_url": os.environ.get("LLM_BASE_URL", "https://api.openai.com/v1"),
    "model": os.environ.get("LLM_MODEL", "gpt-4o-mini"),
}

Do not read process.env inside module top-level code that runs at import time in a bundler that replaces process.env.X with a string. That bakes the local value (or undefined) into the artifact and ships it.

Step 3: Separate build-time from runtime configuration

Vite, Next.js, and esbuild inline process.env.NODE_ENV and any VITE_/NEXT_PUBLIC_ prefixed vars at build time. Custom vars like LLM_API_KEY are not inlined unless you explicitly configure them.

Bad pattern that causes env variable errors serverless llm deploy:

// bundled at build time, value frozen
const client = new OpenAI({ apiKey: process.env.LLM_API_KEY });
export async function handler() {
  return client.chat.completions.create({ model: 'gpt-4o' });
}

Good pattern: construct the client inside the handler or pass vars explicitly:

export async function handler() {
  const client = new OpenAI({
    apiKey: process.env.LLM_API_KEY,
    baseURL: process.env.LLM_BASE_URL,
  });
  return client.chat.completions.create({ model: process.env.LLM_MODEL });
}

This adds a few milliseconds of client init per cold start, which is cheaper than a broken deploy. For Python Lambda, the same rule applies: instantiate the SDK client inside the handler, not at module scope, unless you are certain the vars exist at build.

Step 4: Confirm platform-specific env injection

Each platform stores vars differently. Verify they actually reached the function.

Vercel

vercel env ls
vercel env pull .env.local  # local only, not used in prod

Google Cloud Functions

gcloud functions describe my-llm-fn \
  --format='get(envVariables)'

AWS Lambda (Terraform audit)

resource "aws_lambda_function" "llm" {
  # ...
  environment {
    variables = {
      LLM_API_KEY = var.llm_key
      LLM_BASE_URL = "https://api.openai.com/v1"
    }
  }
}

Run terraform plan to confirm the variable block is not being overridden by a later null assignment.

Cloudflare Workers Secrets are not env vars; they go through wrangler secret put and appear as env bindings:

wrangler secret list

If a var is missing in the describe output, the console UI lied or you deployed an old revision. Redeploy explicitly with --env-vars-file (GCP) or update the Lambda configuration via CI, not the web console.

Step 5: Fetch secrets asynchronously inside the handler

Hard-coding keys in env vars is fine for low-risk apps, but secret managers are common. The mistake is fetching once at module load:

// WRONG: runs at cold start before secrets resolved
const secret = await getSecret('llm-key'); // top-level await hangs

Do it inside the handler and cache across warm invocations:

import { SecretsManager } from '@aws-sdk/client-secrets-manager';
const sm = new SecretsManager({});

let cachedKey;
export async function handler(event) {
  if (!cachedKey) {
    const resp = await sm.getSecretValue({ SecretId: 'llm-api-key' });
    cachedKey = JSON.parse(resp.SecretString).key;
  }
  const client = new OpenAI({ apiKey: cachedKey, baseURL: process.env.LLM_BASE_URL });
  // ...
}

For Google Secret Manager:

from google.cloud import secretmanager

def get_key():
    client = secretmanager.SecretManagerServiceClient()
    name = "projects/123/secrets/llm-key/versions/latest"
    return client.access_secret_version(name=name).payload.data.decode()

cached = None
def handler(request):
    global cached
    if not cached:
        cached = get_key()

Cache the value in a module-scoped variable so subsequent warm invocations skip the round trip. Rotation requires a new deploy or a TTL on the cache.

Step 6: Fail fast with a validation guard

A missing var should throw a loud, specific error on first invocation, not a generic upstream 401. Add a boot check:

function assertEnv() {
  const required = ['LLM_API_KEY', 'LLM_BASE_URL'];
  const missing = required.filter(k => !process.env[k]);
  if (missing.length) {
    throw new Error(`Missing env vars: ${missing.join(', ')}`);
  }
}

export async function handler(event) {
  assertEnv();
  // proceed
}

In Python:

import os

def assert_env():
    missing = [k for k in ("LLM_API_KEY", "LLM_BASE_URL") if not os.environ.get(k)]
    if missing:
        raise RuntimeError(f"Missing env vars: {','.join(missing)}")

This turns a confusing model client stack trace into a one-line deploy log. Redact the values in any logging—never print LLM_API_KEY even partially.

Step 7: Deploy a probe route to verify live

Before wiring the real LLM call, ship a stub that echoes masked env state:

export async function handler() {
  return {
    statusCode: 200,
    body: JSON.stringify({
      hasKey: !!process.env.LLM_API_KEY,
      baseUrl: process.env.LLM_BASE_URL,
      model: process.env.LLM_MODEL ?? 'default',
    }),
  };
}

Hit it:

curl -s https://my-fn.example.com/probe | jq

You should see hasKey: true and the expected base URL. If not, stop and fix the platform config. For regional deployments, run the probe in each region—env vars are sometimes set per-region and missed in us-east-2.

Step 8: Reduce the variable count with a unified gateway

Every additional provider key is another chance for env variable errors serverless llm deploy. If you route through a single OpenAI-compatible gateway such as n4n.ai, you collapse provider-specific credentials into one LLM_BASE_URL and one LLM_API_KEY. The gateway handles fallback across 240+ models, honors client routing directives, and forwards provider cache-control hints, so you no longer juggle ANTHROPIC_API_KEY, OPENAI_API_KEY, and COHERE_API_KEY in three separate secret stores.

const client = new OpenAI({
  apiKey: process.env.LLM_API_KEY,
  baseURL: process.env.LLM_BASE_URL, // single gateway endpoint
});

That shrinks your env surface from a dozen vars to two and removes an entire class of misconfiguration.

Verify success

After applying the steps, redeploy and call the probe route first. Confirm hasKey is true and baseUrl matches your gateway or provider. Then run a minimal completion:

curl -s https://my-fn.example.com/invoke \
  -H 'content-type: application/json' \
  -d '{"prompt":"say hi"}'

Check CloudWatch or Workers logs for the assertEnv pass and a clean model response. If the probe shows green and the completion returns tokens, the env variable errors serverless llm deploy are resolved. Any subsequent failure will be in prompt logic, not configuration.