Set environment variables for LlamaIndex and n4n.ai

Environment variables are the contract between your application and the infrastructure it runs on. For LlamaIndex applications routing through n4n.ai, getting these right determines whether your requests hit the right models, respect rate limits, and surface usable observability. This guide walks through the complete configuration surface area — what to set, where to set it, and how to verify it works before you ship.

Step 1: Understand the required variables

LlamaIndex reads configuration from environment variables at import time. When you point LlamaIndex at n4n.ai, three variables are non-negotiable:

# Required
OPENAI_API_KEY="sk-n4n-..."          # Your n4n.ai API key
OPENAI_API_BASE="https://api.n4n.ai/v1"  # n4n.ai OpenAI-compatible endpoint
LLAMA_INDEX_DEFAULT_MODEL="meta-llama/llama-3.1-70b-instruct"  # Default model identifier

The OPENAI_API_KEY and OPENAI_API_BASE pair is the standard OpenAI SDK contract. n4n.ai honors this interface, so LlamaIndex’s OpenAI and OpenAILike classes work without wrapper code. The LLAMA_INDEX_DEFAULT_MODEL tells LlamaIndex which model string to pass when you instantiate an LLM without an explicit model argument.

Optional but recommended variables:

# Optional — observability and routing
N4N_ROUTING_TAG="production"         # Custom routing tag for n4n.ai dashboard
N4N_FALLBACK_MODELS="openai/gpt-4o,anthropic/claude-3.5-sonnet"  # Comma-separated fallback chain
LLAMA_INDEX_EMBEDDING_MODEL="nomic-ai/nomic-embed-text-v1.5"  # Default embedding model
HTTP_TIMEOUT="60"                    # Seconds; n4n.ai streams can exceed default 30s

The N4N_ROUTING_TAG and N4N_FALLBACK_MODELS are n4n.ai-specific hints forwarded as headers. They don’t affect LlamaIndex directly but shape how n4n.ai routes and meters your traffic. HTTP_TIMEOUT prevents spurious failures on long generations — n4n.ai streams tokens until completion, which can exceed the default 30-second client timeout.

Step 2: Configure for local development

Shell profile (persistent)

Add to ~/.zshrc, ~/.bashrc, or ~/.config/fish/config.fish:

# n4n.ai + LlamaIndex
export OPENAI_API_KEY="sk-n4n-..."
export OPENAI_API_BASE="https://api.n4n.ai/v1"
export LLAMA_INDEX_DEFAULT_MODEL="meta-llama/llama-3.1-70b-instruct"
export LLAMA_INDEX_EMBEDDING_MODEL="nomic-ai/nomic-embed-text-v1.5"
export HTTP_TIMEOUT="60"

Reload with source ~/.zshrc (or your shell’s equivalent).

`.env` file (project-scoped, gitignored)

Create .env at your project root:

# .env — never commit this
OPENAI_API_KEY=sk-n4n-...
OPENAI_API_BASE=https://api.n4n.ai/v1
LLAMA_INDEX_DEFAULT_MODEL=meta-llama/llama-3.1-70b-instruct
LLAMA_INDEX_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5
HTTP_TIMEOUT=60
N4N_ROUTING_TAG=dev
N4N_FALLBACK_MODELS=openai/gpt-4o,anthropic/claude-3.5-sonnet

Load it in Python before importing LlamaIndex:

# load_env.py — run this first, or import at top of entrypoint
from dotenv import load_dotenv
load_dotenv()  # reads .env into os.environ

# Now safe to import LlamaIndex
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

Install python-dotenv if missing: pip install python-dotenv.

Direnv (automatic per-directory)

If you use direnv, create .envrc:

# .envrc
dotenv .env

Run direnv allow once. Variables load automatically when you cd into the project.

Step 3: Configure for Docker

Dockerfile (build-time defaults, override at runtime)

# Dockerfile
FROM python:3.11-slim

WORKDIR /app

# Build-time defaults — override with --build-arg or runtime -e
ARG OPENAI_API_BASE=https://api.n4n.ai/v1
ARG LLAMA_INDEX_DEFAULT_MODEL=meta-llama/llama-3.1-70b-instruct
ARG LLAMA_INDEX_EMBEDDING_MODEL=nomic-ai/nomic-embed-text-v1.5
ARG HTTP_TIMEOUT=60

ENV OPENAI_API_BASE=${OPENAI_API_BASE} \
    LLAMA_INDEX_DEFAULT_MODEL=${LLAMA_INDEX_DEFAULT_MODEL} \
    LLAMA_INDEX_EMBEDDING_MODEL=${LLAMA_INDEX_EMBEDDING_MODEL} \
    HTTP_TIMEOUT=${HTTP_TIMEOUT}

# Never bake secrets into images
# OPENAI_API_KEY must be provided at runtime

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "main.py"]

Docker-compose.yml (local stack)

# docker-compose.yml
version: "3.8"
services:
  app:
    build: .
    env_file:
      - .env          # loads OPENAI_API_KEY, N4N_ROUTING_TAG, etc.
    environment:
      - OPENAI_API_BASE=https://api.n4n.ai/v1
      - LLAMA_INDEX_DEFAULT_MODEL=meta-llama/llama-3.1-70b-instruct
      - HTTP_TIMEOUT=60
    ports:
      - "8000:8000"
    depends_on:
      - redis
  redis:
    image: redis:7-alpine

The env_file directive loads .env (gitignored) into the container. Runtime environment keys override or supplement.

Runtime override (production)

docker run --rm \
  -e OPENAI_API_KEY="${PROD_N4N_KEY}" \
  -e N4N_ROUTING_TAG="production" \
  -e N4N_FALLBACK_MODELS="openai/gpt-4o,anthropic/claude-3.5-sonnet" \
  my-llamaindex-app:latest

Never commit OPENAI_API_KEY to image layers. Pass it at runtime via your orchestrator’s secret store (Kubernetes secrets, AWS Secrets Manager, Doppler, etc.).

Step 4: Configure for CI/CD pipelines

GitHub Actions

# .github/workflows/test.yml
name: Test
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    env:
      OPENAI_API_BASE: https://api.n4n.ai/v1
      LLAMA_INDEX_DEFAULT_MODEL: meta-llama/llama-3.1-70b-instruct
      LLAMA_INDEX_EMBEDDING_MODEL: nomic-ai/nomic-embed-text-v1.5
      HTTP_TIMEOUT: "60"
      N4N_ROUTING_TAG: ci
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run tests
        env:
          OPENAI_API_KEY: ${{ secrets.N4N_API_KEY }}
        run: pytest -v

Store N4N_API_KEY in GitHub repository settings → Secrets → Actions. The env block at job level sets defaults; the step-level env injects the secret only for the test run.

GitLab CI

# .gitlab-ci.yml
variables:
  OPENAI_API_BASE: "https://api.n4n.ai/v1"
  LLAMA_INDEX_DEFAULT_MODEL: "meta-llama/llama-3.1-70b-instruct"
  HTTP_TIMEOUT: "60"
  N4N_ROUTING_TAG: "ci"

test:
  stage: test
  image: python:3.11-slim
  before_script:
    - pip install -r requirements.txt
  script:
    - pytest -v
  variables:
    OPENAI_API_KEY: $N4N_API_KEY  # defined in Settings → CI/CD → Variables (masked)

Step 5: Configure for production (Kubernetes)

ConfigMap (non-secret config)

# k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: llamaindex-config
  namespace: production
data:
  OPENAI_API_BASE: "https://api.n4n.ai/v1"
  LLAMA_INDEX_DEFAULT_MODEL: "meta-llama/llama-3.1-70b-instruct"
  LLAMA_INDEX_EMBEDDING_MODEL: "nomic-ai/nomic-embed-text-v1.5"
  HTTP_TIMEOUT: "60"
  N4N_ROUTING_TAG: "production"
  N4N_FALLBACK_MODELS: "openai/gpt-4o,anthropic/claude-3.5-sonnet"

Secret (API key)

# k8s/secret.yaml — apply via kubectl apply -f secret.yaml
# Or use sealed-secrets, external-secrets, or your GitOps tool
apiVersion: v1
kind: Secret
metadata:
  name: n4n-credentials
  namespace: production
type: Opaque
stringData:
  OPENAI_API_KEY: "sk-n4n-..."

Deployment (wire them together)

# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llamaindex-app
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: llamaindex-app
  template:
    metadata:
      labels:
        app: llamaindex-app
    spec:
      containers:
        - name: app
          image: my-org/llamaindex-app:v1.2.3
          envFrom:
            - configMapRef:
                name: llamaindex-config
            - secretRef:
                name: n4n-credentials
          ports:
            - containerPort: 8000
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10

The envFrom directive merges ConfigMap and Secret into the container’s environment. LlamaIndex picks them up at import time — no code changes needed.

Step 6: Verify the configuration works

Quick smoke test (Python REPL)

python -c "
import os
from llama_index.llms.openai import OpenAI

# Verify env vars are visible
print('OPENAI_API_BASE:', os.getenv('OPENAI_API_BASE'))
print('DEFAULT_MODEL:', os.getenv('LLAMA_INDEX_DEFAULT_MODEL'))

# Verify LlamaIndex picks them up
llm = OpenAI()  # no args — uses env vars
print('LLM model:', llm.model)
print('LLM api_base:', llm.api_base)

# Actual request (will hit n4n.ai)
resp = llm.complete('Say hello in one word.')
print('Response:', resp.text)
"

Expected output:

OPENAI_API_BASE: https://api.n4n.ai/v1
DEFAULT_MODEL: meta-llama/llama-3.1-70b-instruct
LLM model: meta-llama/llama-3.1-70b-instruct
LLM api_base: https://api.n4n.ai/v1
Response: Hello

If llm.model shows gpt-3.5-turbo or llm.api_base shows https://api.openai.com/v1, your environment variables aren’t loading before LlamaIndex imports. Move load_dotenv() or shell export earlier in your entrypoint.

Health endpoint (for orchestration)

Add a /health route that exercises the full stack:

# health.py
from fastapi import APIRouter, HTTPException
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
import os

router = APIRouter()

@router.get("/health")
async def health_check():
    try:
        # Verify config
        api_base = os.getenv("OPENAI_API_BASE")
        model = os.getenv("LLAMA_INDEX_DEFAULT_MODEL")
        if not api_base or not model:
            raise HTTPException(500, "Missing required env vars")

        # Verify connectivity + auth
        llm = OpenAI()
        await llm.acomplete("health check")

        # Verify embeddings
        embed = OpenAIEmbedding()
        await embed.aget_query_embedding("test")

        return {
            "status": "healthy",
            "config": {
                "api_base": api_base,
                "model": model,
                "embedding_model": os.getenv("LLAMA_INDEX_EMBEDDING_MODEL"),
            }
        }
    except Exception as e:
        raise HTTPException(503, f"Unhealthy: {e}")

Deploy this and point your load balancer’s health check at /health. It validates env vars, network reachability, authentication, and both completion and embedding paths.

Verify n4n.ai routing headers

n4n.ai echoes routing metadata in response headers. Inspect them to confirm your tags and fallback chain are honored:

import httpx
from llama_index.llms.openai import OpenAI

llm = OpenAI()

# Make a request and capture raw response
async def test_routing():
    async with httpx.AsyncClient() as client:
        # n4n.ai adds these headers to responses
        resp = await client.post(
            "https://api.n4n.ai/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.getenv('OPENAI_API_KEY')}"},
            json={
                "model": "meta-llama/llama-3.1-70b-instruct",
                "messages": [{"role": "user", "content": "test"}],
                "max_tokens": 5,
            },
        )
        print("x-n4n-model-used:", resp.headers.get("x-n4n-model-used"))
        print("x-n4n-routing-tag:", resp.headers.get("x-n4n-routing-tag"))
        print("x-n4n-fallback-attempted:", resp.headers.get("x-n4n-fallback-attempted"))

import asyncio
asyncio.run(test_routing())

Output like x-n4n-model-used: meta-llama/llama-3.1-70b-instruct and x-n4n-routing-tag: production confirms your N4N_ROUTING_TAG and model selection propagate correctly. If x-n4n-fallback-attempted: true appears, your primary model was unavailable and n4n.ai fell back to the chain in N4N_FALLBACK_MODELS.

Step 7: Common failure modes and fixes

Symptom	Cause	Fix
`AuthenticationError: Invalid API key`	`OPENAI_API_KEY` not set or wrong prefix	Ensure key starts with `sk-n4n-`; verify secret injection in CI/K8s
`ConnectError: api.openai.com`	`OPENAI_API_BASE` not loaded before LlamaIndex import	Move `load_dotenv()` to top of entrypoint; check shell profile reload
`TimeoutError` after 30s	Default `HTTP_TIMEOUT` too low for streaming	Set `HTTP_TIMEOUT=60` (or higher) in env
`ModelNotFound: meta-llama/llama-3.1-70b-instruct`	Model identifier typo or not available in your n4n.ai plan	List available models via `GET /v1/models` on n4n.ai endpoint
Embeddings fail with 404	`LLAMA_INDEX_EMBEDDING_MODEL` points to chat model	Use embedding model ID (e.g., `nomic-ai/nomic-embed-text-v1.5`)
Fallback never triggers	`N4N_FALLBACK_MODELS` format wrong	Comma-separated, no spaces: `openai/gpt-4o,anthropic/claude-3.5-sonnet`

Step 8: Per-request overrides (when you need them)

Environment variables set defaults. For multi-tenant apps or A/B tests, override per-request:

from llama_index.llms.openai import OpenAI

# Default (uses env vars)
default_llm = OpenAI()

# Override model for this instance
coding_llm = OpenAI(model="anthropic/claude-3.5-sonnet")

# Override base URL for direct provider access (bypass n4n.ai)
direct_llm = OpenAI(
    api_base="https://api.anthropic.com/v1",
    api_key=os.getenv("ANTHROPIC_API_KEY"),
    model="claude-3.5-sonnet",
)

# Override routing tag for a specific workflow
import httpx
custom_client = httpx.AsyncClient(
    headers={"X-N4N-Routing-Tag": "premium-tier"}
)
premium_llm = OpenAI(http_client=custom_client)

The http_client injection is the cleanest way to pass n4n.ai-specific headers without forking LlamaIndex.

Summary checklist

Before merging or deploying, confirm:

OPENAI_API_KEY loaded from secret store (never in repo, never in image)
OPENAI_API_BASE=https://api.n4n.ai/v1 set in all environments
LLAMA_INDEX_DEFAULT_MODEL matches a model available in your n4n.ai plan
HTTP_TIMEOUT=60 (or higher) prevents streaming truncation
N4N_ROUTING_TAG distinguishes environments (dev, staging, production)
N4N_FALLBACK_MODELS defines a sensible degradation path
Smoke test passes: python -c "from llama_index.llms.openai import OpenAI; print(OpenAI().complete('ok').text)"
Health endpoint returns 200 with correct config echo
Response headers show expected x-n4n-model-used and x-n4n-routing-tag

Environment configuration is the least glamorous part of shipping LLM features, but it’s the one that bites you at 2 AM when a fallback doesn’t trigger or a staging key leaks into production. Set it once, verify it programmatically, and treat changes like schema migrations — reviewed, tested, and deployed with intent.