Serverless RAG pipelines with AWS Lambda and n4n.ai

A well-designed aws lambda serverless rag pipeline keeps retrieval and generation fully managed, scaling to zero when idle and to thousands of concurrent requests under load. This tutorial builds one from scratch: document embeddings indexed in FAISS, stored in S3, and a Lambda function that answers questions through an OpenAI-compatible LLM endpoint. You’ll get runnable Python, IAM notes, and real output snippets at each checkpoint.

Prerequisites

• AWS account with permissions to create Lambda functions, S3 buckets, IAM roles, and publish layers.
• Python 3.12 local environment with pip and awscli configured.
• An API key for an OpenAI-compatible gateway. We use n4n.ai, which fronts 240+ models and fails over automatically when a provider is rate-limited.
• Familiarity with Lambda layers and basic boto3 usage.

Step 1: Generate and upload the vector index

Run this locally to embed a small corpus and write a FAISS index to S3. Use the same embedding model you’ll call from Lambda.

# build_index.py
import io, json, boto3
import numpy as np
import faiss
from openai import OpenAI

client = OpenAI(api_key="YOUR_KEY", base_url="https://api.n4n.ai/v1")

docs = [
    "Lambda cold starts happen when a new execution context spins up.",
    "FAISS is a library for efficient similarity search on dense vectors.",
    "Serverless RAG avoids managing always-on retrieval servers.",
    # ... pad to your real corpus
]

embs = []
for d in docs:
    r = client.embeddings.create(model="text-embedding-3-small", input=d)
    embs.append(r.data[0].embedding)

matrix = np.array(embs, dtype=np.float32)
index = faiss.IndexFlatL2(matrix.shape[1])
index.add(matrix)

buf = io.BytesIO()
faiss.write_index(index, buf)
s3 = boto3.client("s3")
s3.put_object(Bucket="my-rag-bucket", Key="index/faiss.index", Body=buf.getvalue())
s3.put_object(Bucket="my-rag-bucket", Key="index/faiss.index.txt.json",
              Body=json.dumps(docs))
print(f"Indexed {len(docs)} docs, uploaded to s3://my-rag-bucket/index/")

Expected output:

Indexed 42 docs, uploaded to s3://my-rag-bucket/index/

Step 2: Package Lambda dependencies

Lambda’s runtime doesn’t include faiss-cpu, numpy, or openai. Build a layer:

mkdir lambda_layer && cd lambda_layer
pip install --target . faiss-cpu numpy openai
zip -r ../rag-layer.zip .
aws lambda publish-layer-version \
  --layer-name rag-deps \
  --zip-file fileb://../rag-layer.zip

Note the returned LayerVersionArn for the next step.

Step 3: Implement the handler

Write handler.py. The index and document texts are loaded once per execution environment and reused across invocations.

import os, io, json
import boto3
import numpy as np
import faiss
from openai import OpenAI

s3 = boto3.client("s3")
BUCKET = os.environ["INDEX_BUCKET"]
KEY = os.environ["INDEX_KEY"]
EMBED_MODEL = "text-embedding-3-small"
LLM_MODEL = "gpt-4o-mini"

client = OpenAI(api_key=os.environ["API_KEY"], base_url="https://api.n4n.ai/v1")

_index = None
_texts = None

def load_artifacts():
    global _index, _texts
    if _index is None:
        obj = s3.get_object(Bucket=BUCKET, Key=KEY)
        _index = faiss.read_index(io.BytesIO(obj["Body"].read()))
        txt_obj = s3.get_object(Bucket=BUCKET, Key=KEY + ".txt.json")
        _texts = json.loads(txt_obj["Body"].read())
    return _index, _texts

def embed(text):
    resp = client.embeddings.create(model=EMBED_MODEL, input=text)
    return np.array(resp.data[0].embedding, dtype=np.float32)

def handler(event, context):
    q = json.loads(event["body"])["question"]
    idx, texts = load_artifacts()
    q_emb = embed(q).reshape(1, -1)
    _, I = idx.search(q_emb, 3)
    ctx = "\n".join(texts[i] for i in I[0])
    completion = client.chat.completions.create(
        model=LLM_MODEL,
        messages=[
            {"role": "system", "content": "Answer strictly using the context."},
            {"role": "user", "content": f"Context:\n{ctx}\n\nQuestion: {q}"},
        ],
    )
    return {
        "statusCode": 200,
        "body": json.dumps({"answer": completion.choices[0].message.content}),
    }

Zip the handler and deploy:

zip function.zip handler.py
aws lambda create-function \
  --function-name rag-query \
  --runtime python3.12 \
  --handler handler.handler \
  --role arn:aws:iam::123456789012:role/lambda-rag \
  --zip-file fileb://function.zip \
  --layers arn:aws:lambda:us-east-1:123456789012:layer:rag-deps:1 \
  --environment Variables="{INDEX_BUCKET=my-rag-bucket,INDEX_KEY=index/faiss.index,API_KEY=YOUR_KEY}" \
  --timeout 30 \
  --memory-size 512

Step 4: Test the function

Invoke with a test event:

{"body": "{\"question\": \"How do I reduce Lambda cold starts for ML?\"}"}

Expected response body (truncated):

{"statusCode": 200, "body": "{\"answer\": \"Use provisioned concurrency or load models outside the handler. In this aws lambda serverless rag pipeline the FAISS index is cached in global scope so only the first invoke pays the S3 fetch cost.\"}"}

If you see UnableToImportModule, the layer architecture mismatches the runtime—rebuild on Amazon Linux 2023.

Step 5: Expose via API Gateway

Create a Lambda proxy integration on a new REST API. Map POST /ask to rag-query. The aws lambda serverless rag pipeline now accepts HTTP traffic without any always-on infrastructure.

Operational notes for production

Cold starts dominate the first request per worker. Keep the index small (<50 MB) so S3 fetch stays under 200 ms, or mount it on EFS for larger corpora. The embedding and chat calls go through the OpenAI-compatible endpoint; n4n.ai also meters per-token usage and honors cache-control hints, which keeps cost predictable when traffic spikes.

Set LLM_MODEL and EMBED_MODEL via environment variables so you can swap models without redeploying. Because the gateway fronts many providers, a degraded upstream silently falls back—your Lambda code doesn’t need retry logic for provider 429s.

For concurrency above 1,000, raise the Lambda account limit and use S3 eventual consistency wisely: publish a new index version under a distinct key and bump INDEX_KEY atomically. The aws lambda serverless rag pipeline described here avoids stateful servers entirely, but you should still add a dead-letter queue for malformed events.

Finally, log the completion.usage object. In a real workload that reveals whether your retrieval prompt is bloating token spend—trim context to the top-2 matches if you see steady growth. That’s the difference between a demo and a deployable aws lambda serverless rag pipeline.