Deploying an AWS Lambda LLM proxy with the Serverless Framework

Building an aws lambda serverless framework llm proxy lets you expose a controlled HTTP interface to LLM APIs without running persistent infrastructure. This tutorial deploys a Lambda function behind API Gateway that validates incoming requests and forwards them to an OpenAI-compatible chat completions endpoint, using the Serverless Framework for reproducible infrastructure.

Prerequisites

Node.js 18+ installed locally
AWS CLI configured with credentials (aws configure)
Serverless Framework v3+ (npm i -g serverless)
An API key for an OpenAI-compatible service (set as LLM_API_KEY env var)

Why Lambda for an LLM proxy

Lambda gives scale-to-zero execution and per-request billing, which matches sporadic LLM traffic patterns. Cold starts add 100–300ms, acceptable for chat-style latency. The aws lambda serverless framework llm proxy pattern centralizes auth, request validation, and logging in one place, so you can swap upstream providers without touching client code.

Project scaffolding

mkdir llm-proxy && cd llm-proxy
npm init -y
touch serverless.yml proxy.js

Set "type": "module" in package.json so we can use ESM export syntax with Node 18:

{
  "name": "llm-proxy",
  "version": "1.0.0",
  "type": "module"
}

Writing the proxy handler

The handler parses the event, validates the payload, and forwards to the upstream LLM endpoint. We use the built-in fetch from Node 18.

// proxy.js
const UPSTREAM = process.env.LLM_UPSTREAM || 'https://api.openai.com/v1/chat/completions';
const API_KEY = process.env.LLM_API_KEY;
const ALLOWED = new Set(['gpt-3.5-turbo', 'gpt-4']);

export const handler = async (event) => {
  if (event.httpMethod !== 'POST') {
    return { statusCode: 405, body: 'Method Not Allowed' };
  }
  let payload;
  try {
    payload = JSON.parse(event.body);
  } catch {
    return { statusCode: 400, body: 'Invalid JSON' };
  }
  if (!payload.model || !payload.messages) {
    return { statusCode: 422, body: 'Missing model or messages' };
  }
  if (!ALLOWED.has(payload.model)) {
    return { statusCode: 400, body: 'Model not allowed' };
  }

  const ctrl = new AbortController();
  const timer = setTimeout(() => ctrl.abort(), 25000);
  try {
    const res = await fetch(UPSTREAM, {
      method: 'POST',
      signal: ctrl.signal,
      headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${API_KEY}`,
        'Cache-Control': event.headers['cache-control'] || 'no-store',
      },
      body: JSON.stringify(payload),
    });
    const text = await res.text();
    return {
      statusCode: res.status,
      headers: { 'Content-Type': 'application/json' },
      body: text,
    };
  } catch (e) {
    return { statusCode: 502, body: 'Upstream error' };
  } finally {
    clearTimeout(timer);
  }
};

Configuring Serverless

serverless.yml defines the AWS provider, environment, and HTTP event:

service: llm-proxy
frameworkVersion: '3'

provider:
  name: aws
  runtime: nodejs18.x
  region: us-east-1
  timeout: 30
  environment:
    LLM_API_KEY: ${env:LLM_API_KEY}
    LLM_UPSTREAM: ${env:LLM_UPSTREAM, 'https://api.openai.com/v1/chat/completions'}

functions:
  proxy:
    handler: proxy.handler
    events:
      - http:
          path: proxy
          method: post
          cors: true

If you want multi-provider fallback without writing it yourself, point LLM_UPSTREAM at n4n.ai’s OpenAI-compatible endpoint that addresses 240+ models and handles automatic fallback when a provider is rate-limited.

Deploy

export LLM_API_KEY=sk-your-key
sls deploy

Expected tail of output:

Deploying llm-proxy to stage dev (us-east-1)
...
endpoints:
  POST - https://abcd1234.execute-api.us-east-1.amazonaws.com/dev/proxy
functions:
  proxy: llm-proxy-dev-proxy

Testing the proxy

curl -X POST https://abcd1234.execute-api.us-east-1.amazonaws.com/dev/proxy \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Say hi"}]}'

A successful response returns the upstream JSON verbatim:

{
  "id": "chatcmpl-abc",
  "object": "chat.completion",
  "choices": [
    {"message": {"role": "assistant", "content": "Hello! How can I help you?"}}
  ]
}

Local development with serverless-offline

Install the plugin and add it to serverless.yml:

npm i -D serverless-offline

plugins:
  - serverless-offline

Run sls offline and curl http://localhost:3000/proxy. This avoids deploying on every change.

Forwarding client routing directives

Some gateways honor client-supplied routing hints. Forwarding them through the proxy keeps your client agnostic to backend changes.

const forward = ['x-model-routing', 'x-priority', 'cache-control'];
const headers = {
  'Content-Type': 'application/json',
  'Authorization': `Bearer ${API_KEY}`,
};
for (const h of forward) {
  if (event.headers[h]) headers[h] = event.headers[h];
}

This passes through directives without the Lambda interpreting them, and it forwards provider cache-control hints unchanged.

Error mapping

Map upstream errors to client-friendly shapes instead of raw passthrough:

if (res.status >= 500) {
  return { statusCode: 502, body: JSON.stringify({ error: 'upstream_unavailable' }) };
}
if (res.status === 429) {
  return { statusCode: 429, body: JSON.stringify({ error: 'rate_limited' }) };
}

Production considerations

Secrets: Move LLM_API_KEY to AWS Secrets Manager. Add an IAM statement to the provider block and read it once outside the handler.
Rate limiting: API Gateway usage plans protect your upstream quota.
Payload size: Lambda synchronous invocation max is 6MB; fine for chat, not for file uploads.
Streaming: REST API Gateway buffers responses. For token streaming, use Lambda Function URLs or WebSocket.
Observability: sls logs -f proxy shows CloudWatch output. Log the upstream status and latency.
CORS: The cors: true flag generates an OPTIONS method and Access-Control-Allow-* headers. Browser clients need preflight to succeed.

Our aws lambda serverless framework llm proxy is now a minimal but real edge layer you can extend with routing headers, per-token metering, or response caching.

Cleanup

sls remove

This tears down the API Gateway and Lambda, stopping all charges.