Running Vercel AI SDK on AWS Lambda

The Vercel AI SDK works on AWS Lambda, but the documentation assumes you’re on Vercel’s platform. Running it on Lambda requires a custom handler that bridges Node’s streaming response to Lambda’s event model, plus careful attention to timeout limits and cold-start behavior. This guide walks through a production-ready setup from empty directory to deployed function.

Step 1: Initialize the project and install dependencies

Create a new directory and set up a minimal Node.js project with TypeScript. The AI SDK requires the core package, a provider, and the AWS Lambda adapter types.

mkdir vercel-ai-lambda && cd vercel-ai-lambda
npm init -y
npm install ai @ai-sdk/openai @aws-sdk/client-lambda
npm install -D typescript @types/node @types/aws-lambda tsx

Create a tsconfig.json targeting ES2022 with NodeNext module resolution — this matches the Lambda Node.js 20 runtime.

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "lib": ["ES2022"],
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "declaration": true,
    "resolveJsonModule": true
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "dist"]
}

Step 2: Create the core chat route logic

The AI SDK’s streamText returns a ReadableStream that works natively in edge runtimes. On Lambda, you need to convert that stream into the format API Gateway expects. Create src/chat.ts with the provider-agnostic logic separated from the Lambda handler.

// src/chat.ts
import { streamText, CoreMessage, StreamTextResult } from 'ai';
import { openai } from '@ai-sdk/openai';

export interface ChatRequest {
  messages: CoreMessage[];
  model?: string;
  temperature?: number;
  maxTokens?: number;
}

export async function generateChatResponse(
  request: ChatRequest
): Promise<StreamTextResult> {
  const model = request.model ?? 'gpt-4o-mini';
  
  return streamText({
    model: openai(model),
    messages: request.messages,
    temperature: request.temperature ?? 0.7,
    maxTokens: request.maxTokens ?? 2048,
  });
}

This separation keeps your business logic testable and portable. The provider choice (OpenAI here) is a single line change — swap to Anthropic, Google, or a gateway like n4n.ai by changing the import and model string.

Step 3: Build the Lambda streaming handler

Lambda’s Node.js runtime doesn’t natively support returning a ReadableStream from the handler. API Gateway expects either a complete response object (for non-streaming) or a specific streaming protocol via the Lambda-Proxy integration with chunked transfer encoding. The cleanest approach: use the aws-lambda-streaming-response pattern with a custom Readable that pipes the AI SDK stream to the Lambda response.

Create src/handler.ts:

// src/handler.ts
import { APIGatewayProxyEventV2, APIGatewayProxyResultV2 } from 'aws-lambda';
import { Readable } from 'node:stream';
import { generateChatResponse, ChatRequest } from './chat';

export const handler = async (
  event: APIGatewayProxyEventV2
): Promise<APIGatewayProxyResultV2> => {
  // Handle CORS preflight
  if (event.requestContext.http.method === 'OPTIONS') {
    return corsResponse(204);
  }

  if (event.requestContext.http.method !== 'POST') {
    return corsResponse(405, { error: 'Method not allowed' });
  }

  let body: ChatRequest;
  try {
    body = JSON.parse(event.body ?? '{}');
  } catch {
    return corsResponse(400, { error: 'Invalid JSON body' });
  }

  if (!body.messages || !Array.isArray(body.messages)) {
    return corsResponse(400, { error: 'Missing or invalid messages array' });
  }

  try {
    const result = await generateChatResponse(body);
    
    // Convert AI SDK stream to Node Readable for Lambda streaming
    const stream = Readable.fromWeb(result.toAIStream() as unknown as ReadableStream<Uint8Array>);
    
    return {
      statusCode: 200,
      headers: {
        'Content-Type': 'text/plain; charset=utf-8',
        'Transfer-Encoding': 'chunked',
        'Cache-Control': 'no-cache',
        'Connection': 'keep-alive',
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Headers': 'Content-Type',
        'Access-Control-Allow-Methods': 'POST, OPTIONS',
      },
      body: stream,
      isBase64Encoded: false,
    } as APIGatewayProxyResultV2 & { body: Readable };
  } catch (error) {
    console.error('Chat generation failed:', error);
    return corsResponse(500, { error: 'Internal server error' });
  }
};

function corsResponse(statusCode: number, body?: unknown): APIGatewayProxyResultV2 {
  return {
    statusCode,
    headers: {
      'Access-Control-Allow-Origin': '*',
      'Access-Control-Allow-Headers': 'Content-Type',
      'Access-Control-Allow-Methods': 'POST, OPTIONS',
      'Content-Type': 'application/json',
    },
    body: body ? JSON.stringify(body) : '',
  };
}

Key details: result.toAIStream() returns a Web ReadableStream compatible with the AI SDK’s streaming protocol. Readable.fromWeb() bridges it to Node’s Readable type, which Lambda’s runtime can pipe directly to the response socket when you return it as the body property. The Transfer-Encoding: chunked header tells API Gateway to stream chunks as they arrive rather than buffering.

Step 4: Configure Lambda function settings

Create a template.yaml for AWS SAM (Serverless Application Model) — it’s the most straightforward way to define the function, API Gateway, and permissions in one file. Adjust the timeout and memory based on your model and expected response length.

# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: nodejs20.x
    Architecture: arm64
    Timeout: 60
    MemorySize: 1024
    Environment:
      Variables:
        OPENAI_API_KEY: '{{resolve:secretsmanager:openai-api-key:SecretString:api_key}}'
        NODE_OPTIONS: '--enable-source-maps'

Resources:
  ChatFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: dist/
      Handler: handler.handler
      Policies:
        - AWSSecretsManagerReadWrite
      Events:
        ChatApi:
          Type: HttpApi
          Properties:
            Path: /chat
            Method: post
            Timeout: 60
            PayloadFormatVersion: '2.0'

Outputs:
  ChatApiUrl:
    Value: !Sub 'https://${ServerlessHttpApi}.execute-api.${AWS::Region}.amazonaws.com/chat'

Notes on this configuration:

• ARM64 (Graviton2) reduces cost by ~20% and cold starts are comparable to x86 for Node.js workloads
• Timeout: 60 seconds is the maximum for API Gateway v2; if you need longer, you must use async patterns with WebSockets or polling
• Memory: 1024 MB gives proportional CPU — drop to 512 MB for cheaper cold starts if latency is acceptable
• OPENAI_API_KEY pulled from Secrets Manager at deploy time via the {{resolve:secretsmanager:...}} intrinsic; never hardcode keys

Create the secret before deploying:

aws secretsmanager create-secret \
  --name openai-api-key \
  --secret-string '{"api_key":"sk-your-key-here"}'

Step 5: Add build and deploy scripts

Update package.json with scripts that compile TypeScript, bundle dependencies, and deploy via SAM. The AI SDK and providers are ESM-only, so tsc output works directly without bundling — but you must copy package.json to dist/ so Lambda can resolve dependencies.

{
  "name": "vercel-ai-lambda",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "build": "tsc && cp package.json dist/ && cd dist && npm ci --production",
    "deploy": "sam build && sam deploy --guided",
    "dev": "tsx watch src/handler.ts",
    "test": "node --test dist/**/*.test.js"
  },
  "dependencies": {
    "ai": "^3.4.0",
    "@ai-sdk/openai": "^0.0.0",
    "@aws-sdk/client-lambda": "^3.0.0"
  },
  "devDependencies": {
    "typescript": "^5.0.0",
    "@types/node": "^20.0.0",
    "@types/aws-lambda": "^8.10.0",
    "tsx": "^4.0.0"
  }
}

The npm ci --production inside dist/ installs only runtime dependencies into the deployment artifact. This keeps the package size small — critical for Lambda’s 250 MB unzipped limit.

Step 6: Deploy and verify

Build the project, then deploy with SAM. The first deploy walks you through stack name, region, and confirmation.

npm run build
sam deploy --guided

When prompted, accept defaults or customize:

• Stack name: vercel-ai-lambda
• Region: us-east-1 (or your preference)
• Confirm changes: y
• Allow SAM CLI IAM role creation: y
• Save arguments to samconfig.toml: y

After deployment completes, SAM outputs the API Gateway URL. Test it with curl:

curl -X POST https://<api-id>.execute-api.us-east-1.amazonaws.com/chat \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Say hello in one sentence."}]}'

You should see a streaming response — chunks arriving progressively rather than all at once. The response format follows the AI SDK’s data stream protocol:

data: {"type":"text-delta","text":"Hello"}
data: {"type":"text-delta","text":" there"}
data: {"type":"text-delta","text":"!"}
data: {"type":"finish","finishReason":"stop"}

If you get a complete JSON object instead of chunks, check that API Gateway’s PayloadFormatVersion is 2.0 and the Lambda response includes Transfer-Encoding: chunked.

Step 7: Add observability and error handling

Production workloads need structured logging, metrics, and graceful degradation. Add a lightweight wrapper around the handler that captures latency, token usage, and errors without blocking the stream.

Create src/observability.ts:

// src/observability.ts
import { StreamTextResult } from 'ai';

interface Metrics {
  latencyMs: number;
  promptTokens: number;
  completionTokens: number;
  model: string;
  error?: string;
}

export function withMetrics<T>(
  operation: () => Promise<T>,
  onComplete: (metrics: Metrics) => void
): Promise<T> {
  const start = Date.now();
  let promptTokens = 0;
  let completionTokens = 0;
  let model = 'unknown';
  let error: string | undefined;

  return operation()
    .then((result) => {
      if (result && typeof result === 'object' && 'usage' in result) {
        const usage = (result as StreamTextResult).usage;
        promptTokens = usage?.promptTokens ?? 0;
        completionTokens = usage?.completionTokens ?? 0;
        model = (result as StreamTextResult).model ?? 'unknown';
      }
      return result;
    })
    .catch((err) => {
      error = err.message;
      throw err;
    })
    .finally(() => {
      onComplete({
        latencyMs: Date.now() - start,
        promptTokens,
        completionTokens,
        model,
        error,
      });
    });
}

Update handler.ts to use it:

// In handler.ts, import withMetrics
import { withMetrics } from './observability';

// Wrap the generateChatResponse call
const result = await withMetrics(
  () => generateChatResponse(body),
  (metrics) => {
    console.log(JSON.stringify({
      level: metrics.error ? 'ERROR' : 'INFO',
      message: 'chat_completion',
      ...metrics,
      timestamp: new Date().toISOString(),
    }));
  }
);

This emits structured JSON logs that CloudWatch Logs Insights can query. For example, find p99 latency by model:

fields @timestamp, latencyMs, model
| filter level = "INFO"
| stats p99(latencyMs) by model

Common pitfalls and fixes

Streaming stops after 30 seconds
API Gateway v2 has a hard 30-second idle timeout on the integration. If the model takes longer than 30 seconds to produce the first token, the connection drops. Mitigation: use a smaller/faster model for first-token latency, or implement a WebSocket-based async pattern where Lambda kicks off generation and pushes to a connection manager.

Cold starts add 1-2 seconds
Node.js 20 on ARM64 typically cold-starts in 800-1500 ms with this dependency set. To reduce: enable SnapStart (Java only, not Node), keep memory at 1024 MB+, or use provisioned concurrency for predictable latency. The AI SDK itself adds minimal overhead — most time is in the provider’s first HTTP request.

Token usage not appearing in logs
The usage property on StreamTextResult is only populated after the stream finishes. The withMetrics wrapper above accesses it in the finally block, which runs after the promise settles. If you log inside the streaming loop, usage will be zero.

CORS errors from browser clients
The handler returns CORS headers on all responses, including errors. If you still see CORS failures, verify API Gateway’s CORS configuration isn’t overriding your headers. In the SAM template, the HttpApi event type handles CORS automatically when you return the headers — no separate Cors config needed.

Package size exceeds limit
Run du -sh dist/ after build. If it’s over 200 MB, audit dependencies. The AI SDK core is ~2 MB; providers add 1-5 MB each. Remove unused providers, and consider @vercel/otel only if you need tracing — it adds significant weight.

Local development workflow

Use tsx watch for hot-reloading during development. It compiles TypeScript on the fly and restarts the handler. For API Gateway simulation, run a local proxy:

# Terminal 1: Start the handler
npm run dev

# Terminal 2: Install and run a local API Gateway emulator
npx @aws-sdk/client-lambda-local-invoke --port 3001 --handler dist/handler.handler

Or use sam local start-api which spins up a Docker container mimicking the Lambda environment — slower but more faithful.

sam local start-api --port 3001 --env-vars env.json

Create env.json for local secrets:

{
  "ChatFunction": {
    "OPENAI_API_KEY": "sk-your-local-key"
  }
}

Scaling considerations

Lambda scales automatically, but the AI provider’s rate limits are the real bottleneck. OpenAI’s tiered limits (RPM, TPM) apply per organization, not per Lambda instance. If you expect burst traffic, implement client-side exponential backoff and consider a queue-based architecture: API Gateway → SQS → Lambda workers → provider. This decouples request acceptance from provider capacity.

For multi-model routing or fallback across providers, the pattern is the same — swap the provider in chat.ts based on request parameters or availability signals. A gateway that normalizes 240+ models behind one OpenAI-compatible endpoint simplifies this to a single model string change.

Deploy complete. You now have a streaming chat endpoint on AWS Lambda using the Vercel AI SDK, with structured logging, secrets management, and a deployment pipeline. The same pattern extends to completions, embeddings, and tool-calling workflows — just swap the AI SDK function and adjust the response streaming logic.