How environment variables protect your LLM API keys

Environment variables API keys remain the baseline defense for keeping secrets out of source control, yet teams still leak credentials through debug logs, Docker layers, and misconfigured CI pipelines. This guide walks through a layered approach that works from laptop to production, with concrete patterns you can adopt today.

Why environment variables beat the alternatives

Hardcoding keys in source files is the obvious anti-pattern — git history never forgets. Config files checked into version control share the same problem. Secrets managers (AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager) solve the problem at scale but add operational overhead that slows down early-stage development. Environment variables sit in the sweet spot: universally supported, zero dependencies, and compatible with every deployment target from a developer’s shell to Kubernetes.

The protection model is straightforward. The application reads process.env.OPENAI_API_KEY (Node) or os.getenv("OPENAI_API_KEY") (Python) at runtime. The value never touches disk in the repository. Your shell, CI system, or orchestrator injects it at startup. If an attacker gains read access to your codebase, they find references to the variable name — not the secret itself.

Local development: keep secrets off the filesystem

Use a `.env` file — but never commit it

Create a .env file in your project root:

# .env
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

Add .env to .gitignore immediately:

# .gitignore
.env
.env.local
.env.*.local

Load it in your application entry point. In Node with dotenv:

// index.js — first line before any other imports
require('dotenv').config();

// Now process.env.OPENAI_API_KEY is available
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

In Python with python-dotenv:

# main.py
from dotenv import load_dotenv
load_dotenv()  # reads .env into os.environ

import os
from openai import OpenAI

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

Pitfall: `.env` files in Docker build context

A common mistake: copying .env into the image via COPY . . in the Dockerfile. The secret becomes baked into a layer. Instead, use --env-file at runtime:

# Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "main.py"]

# Build without secrets
docker build -t my-llm-app .

# Run with secrets injected at container start
docker run --env-file .env my-llm-app

For docker-compose, the env_file directive handles this cleanly:

# docker-compose.yml
services:
  app:
    build: .
    env_file:
      - .env

Pitfall: shell history leakage

Exporting keys directly in your terminal writes them to ~/.bash_history or ~/.zsh_history:

# Bad — ends up in history
export OPENAI_API_KEY=sk-...

Prefer one of these approaches:

# Option 1: Read from a file (not in history)
export OPENAI_API_KEY=$(cat ~/.secrets/openai_key)

# Option 2: Use a tool like direnv — loads .envrc automatically on cd
# .envrc (add to .gitignore)
export OPENAI_API_KEY=sk-...

# Install direnv, then allow the project
direnv allow

Direnv also supports dotenv natively via dotenv .env in .envrc, keeping the same file format across tools.

CI/CD: inject secrets at pipeline runtime

GitHub Actions

Store secrets in the repository settings (Settings → Secrets and variables → Actions), then reference them in workflow files:

# .github/workflows/test.yml
name: Test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install -r requirements.txt
      - run: pytest

The secrets are masked in logs automatically. Never echo them for debugging.

GitLab CI/CD

Similar pattern in .gitlab-ci.yml:

variables:
  OPENAI_API_KEY: $OPENAI_API_KEY  # defined in Settings → CI/CD → Variables

test:
  script:
    - pip install -r requirements.txt
    - pytest

Mark variables as “Protected” and “Masked” in the UI.

CircleCI

Use contexts or project environment variables. In .circleci/config.yml:

version: 2.1
jobs:
  test:
    docker:
      - image: cimg/python:3.11
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY}
    steps:
      - checkout
      - run: pip install -r requirements.txt
      - run: pytest
workflows:
  test:
    jobs:
      - test:
          context: llm-secrets  # defined in Organization Settings → Contexts

Container orchestration: Kubernetes, ECS, and beyond

Kubernetes secrets

Create a secret from literal values or a file:

# Imperative (quick)
kubectl create secret generic llm-keys \
  --from-literal=OPENAI_API_KEY=sk-... \
  --from-literal=ANTHROPIC_API_KEY=sk-ant-...

# Declarative (GitOps-friendly) — use sealed-secrets or external-secrets in practice
kubectl create secret generic llm-keys \
  --from-file=.env \
  --dry-run=client -o yaml > secret.yaml

Reference in your deployment:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llm-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: llm-app
  template:
    metadata:
      labels:
        app: llm-app
    spec:
      containers:
        - name: app
          image: my-registry/llm-app:latest
          envFrom:
            - secretRef:
                name: llm-keys
          # Or individual env vars:
          env:
            - name: OPENAI_API_KEY
              valueFrom:
                secretKeyRef:
                  name: llm-keys
                  key: OPENAI_API_KEY

AWS ECS task definitions

In the task definition JSON or via the console, define environment variables that pull from Secrets Manager or Parameter Store:

{
  "family": "llm-app",
  "containerDefinitions": [
    {
      "name": "app",
      "image": "123456789.dkr.ecr.us-east-1.amazonaws.com/llm-app:latest",
      "secrets": [
        {
          "name": "OPENAI_API_KEY",
          "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789:secret:llm/openai-key:API_KEY::"
        },
        {
          "name": "ANTHROPIC_API_KEY",
          "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789:secret:llm/anthropic-key:API_KEY::"
        }
      ]
    }
  ]
}

The ECS agent injects these as environment variables at container startup. The application code remains unchanged.

Runtime hygiene: prevent accidental exposure

Never log the key

A surprising number of leaks happen through debug logging. Guard against it:

# Python — sanitize before logging
import os
import logging

logger = logging.getLogger(__name__)

def log_config():
    # Safe: log presence, not value
    logger.info("OpenAI configured: %s", bool(os.getenv("OPENAI_API_KEY")))
    # Dangerous: never do this
    # logger.debug("OpenAI key: %s", os.getenv("OPENAI_API_KEY"))

// Node — structured logging with redaction
const pino = require('pino');
const logger = pino({
  redact: {
    paths: ['*.apiKey', '*.authorization', '*.OPENAI_API_KEY'],
    censor: '**REDACTED**'
  }
});

logger.info({ apiKey: process.env.OPENAI_API_KEY }, 'config loaded');
// Output: {"level":30,"time":...,"apiKey":"**REDACTED**","msg":"config loaded"}

Redact in error reporting

Sentry, Datadog, and similar tools capture context automatically. Configure scrubbing:

# sentry_sdk.init(
#     before_send=lambda event, hint: scrub_secrets(event, hint)
# )

def scrub_secrets(event, hint):
    if 'request' in event and 'env' in event['request']:
        env = event['request']['env']
        for key in list(env.keys()):
            if 'KEY' in key.upper() or 'SECRET' in key.upper() or 'TOKEN' in key.upper():
                env[key] = '[REDACTED]'
    return event

Validate presence at startup

Fail fast if a required key is missing — don’t wait for the first API call:

# config.py
import os
import sys

REQUIRED_KEYS = [
    "OPENAI_API_KEY",
    "ANTHROPIC_API_KEY",
]

def validate_env():
    missing = [k for k in REQUIRED_KEYS if not os.getenv(k)]
    if missing:
        sys.stderr.write(f"Missing required environment variables: {', '.join(missing)}\n")
        sys.exit(1)

validate_env()

// config.js
const REQUIRED_KEYS = ['OPENAI_API_KEY', 'ANTHROPIC_API_KEY'];

function validateEnv() {
  const missing = REQUIRED_KEYS.filter(k => !process.env[k]);
  if (missing.length > 0) {
    console.error(`Missing required environment variables: ${missing.join(', ')}`);
    process.exit(1);
  }
}

validateEnv();

Rotating keys without downtime

Environment variables make rotation straightforward: update the value in your secret store or CI settings, then restart or reload the application. For zero-downtime rotation:

Add the new key alongside the old one (e.g., OPENAI_API_KEY_V2)
Deploy a version that reads from both, preferring the new key
Verify traffic uses the new key via provider usage dashboards
Revoke the old key
Clean up the old variable name in a follow-up deploy

# Graceful rotation pattern
def get_openai_key():
    # Prefer new key, fall back to old
    return os.getenv("OPENAI_API_KEY_V2") or os.getenv("OPENAI_API_KEY")

Kubernetes rolling restarts handle this naturally when you update the Secret:

# Update the secret
kubectl create secret generic llm-keys \
  --from-literal=OPENAI_API_KEY=sk-new... \
  --dry-run=client -o yaml | kubectl apply -f -

# Trigger rolling restart
kubectl rollout restart deployment/llm-app

Multi-environment strategy

Use distinct keys per environment (dev, staging, prod) with clear naming:

# .env.development
OPENAI_API_KEY=sk-dev-...

# .env.staging
OPENAI_API_KEY=sk-staging-...

# .env.production
OPENAI_API_KEY=sk-prod-...

Load the appropriate file based on NODE_ENV or ENVIRONMENT:

// config.js
const env = process.env.NODE_ENV || 'development';
require('dotenv').config({ path: `.env.${env}` });

// Fallback to base .env for shared values
require('dotenv').config();

# config.py
import os
from dotenv import load_dotenv

env = os.getenv("ENVIRONMENT", "development")
load_dotenv(f".env.{env}")  # environment-specific
load_dotenv(".env")         # shared fallback

In CI/CD, set the environment variable at the job level:

# .github/workflows/staging.yml
jobs:
  deploy-staging:
    environment: staging
    env:
      ENVIRONMENT: staging
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY_STAGING }}

When environment variables aren’t enough

Environment variables API keys work well for most teams, but they have limits:

No built-in audit trail — you don’t know who accessed the value or when
No automatic rotation — you must orchestrate it yourself
Process memory exposure — any code in the process can read process.env
No fine-grained access control — every container in the pod sees the same secrets

For regulated environments or larger teams, layer a secrets manager on top. The application still reads environment variables; an init container or sidecar (like the AWS Secrets Manager CSI driver, HashiCorp Vault Agent Injector, or External Secrets Operator) populates them from the central store at startup. This gives you audit logs, rotation policies, and RBAC without changing application code.

# Kubernetes with External Secrets Operator
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: llm-keys
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secretsmanager
    kind: ClusterSecretStore
  target:
    name: llm-keys
    creationPolicy: Owner
  data:
    - secretKey: OPENAI_API_KEY
      remoteRef:
        key: prod/llm/openai
        property: API_KEY

The deployment spec stays identical — it still references secretRef: llm-keys.

Checklist before you ship

TL;DR

Environment variables API keys are the pragmatic default for LLM credential management. They keep secrets out of git, work everywhere code runs, and require zero infrastructure to start. Pair them with .env files for local development, platform secret stores for CI/CD, and Kubernetes secrets or cloud provider equivalents for production. Add runtime guards — startup validation, log redaction, error scrubbing — and you have a defensible baseline that scales until you need a full secrets manager.