Environment variables vs secret managers for API keys

The argument about environment variables vs secret managers api keys usually starts with someone pasting a key into a shell script and ends with a SOC 2 audit. Both approaches put credentials into a running process, but they differ sharply in rotation, access control, and failure modes. If you are wiring up authentication to a payment API or an LLM gateway, the choice dictates how fast you can rotate after a leak.

Capabilities

Environment variables inject a plaintext string into a process’s memory at spawn. That is the entire feature set. You get no version history, no automatic rotation, no record of which PID read the value, and no way to revoke a single consumer without restarting everything.

Secret managers treat credentials as versioned objects behind an authenticated API. AWS Secrets Manager, Google Secret Manager, Azure Key Vault, and HashiCorp Vault all support labeling, rolling to a new version without changing code, and attaching IAM policies that restrict GetSecret to a specific role. Vault adds dynamic secrets that expire after minutes.

For a service that needs one static key, the difference is cosmetic. For a fleet of workers calling dozens of third-party APIs, the difference is whether you can rotate a leaked key in seconds or spend an afternoon redeploying.

# Env var: available immediately, no I/O
import os
api_key = os.environ["STRIPE_API_KEY"]

# Secret manager: explicit fetch, auth, and parsing
import boto3, json
sm = boto3.client("secretsmanager", region_name="us-east-1")
raw = sm.get_secret_value(SecretId="prod/stripe")["SecretString"]
api_key = json.loads(raw)["STRIPE_API_KEY"]

A secret manager also lets you bundle related credentials in one JSON blob:

{
  "openai": "sk-...",
  "anthropic": "sk-ant-...",
  "database_url": "postgres://..."
}

Price / Cost Model

Environment variables cost nothing. They ride along with the compute you already pay for; there is no separate meter.

Secret managers bill per secret per month plus per API call. AWS Secrets Manager publishes $0.40 per secret per month and $0.05 per 10,000 API calls. GCP and Azure price similarly—fractions of a cent per version plus request fees. If you store 200 keys and fetch each once per process start, the monthly charge is under a dollar plus negligible call volume. Fetch every request across a high-QPS service and the call fees climb, though they rarely exceed the engineering cost of building your own rotation system.

Self-hosted Vault removes per-secret fees but adds infrastructure and on-call ownership. That is a real cost, just not a line-item one.

Latency / Throughput

An environment variable lookup is a memory read measured in nanoseconds. It never fails due to network partitions.

A secret manager requires a TLS call to a regional endpoint. In the same region, plan on 10–50 ms per fetch. If you call it on every inbound request, you add that latency to every call and create a new failure mode: the manager’s API rate limit. Most clouds allow thousands of requests per second per account, but a misconfigured loop can throttle your own service.

The standard fix is to fetch once at boot or cache with a short TTL:

// Node: cache secret for 5 minutes
let cache: { value: string; expires: number } | null = null;
async function getKey() {
  if (cache && cache.expires > Date.now()) return cache.value;
  const v = await vault.read("secret/data/prod");
  cache = { value: v.data.key, expires: Date.now() + 300_000 };
  return cache.value;
}

Ergonomics

Environment variables win on day one. A .env file, a docker run -e, or a Kubernetes envFrom does the job with zero new dependencies.

docker run -e STRIPE_API_KEY=$STRIPE_API_KEY myworker

Secret managers demand bootstrap credentials for the manager itself, SDK installation, and error handling for AccessDenied or RateExceeded. In Kubernetes you often run an external-secrets operator to sync manager values into native Secret objects, which then become env vars anyway. The ergonomic penalty is front-loaded; the payoff is centralized audit and rotation.

Ecosystem

Environment variables are POSIX. Every language, container runtime, and CI system understands them. No migration risk.

Secret managers are fragmented. AWS Secrets Manager IAM policies do not port to GCP. Vault’s Go SDK is not Azure’s REST contract. If you multi-cloud, you either abstract behind a thin wrapper or accept duplicated logic. For a startup on one provider, this barely matters. For an enterprise with acquisition history, it is a quarterly project.

Limits

Environment variables are constrained by the operating system. Linux caps total environment plus argument size at ARG_MAX (commonly 2 MB) and individual values are practical only up to tens of KB. They are visible in /proc/<pid>/environ to root and anyone with the same UID—plaintext at rest in process memory.

Secret managers impose their own ceilings: AWS limits a secret to 64 KB, and all enforce per-account API quotas. They encrypt at rest and never expose the value in process listings, but a misconfigured IAM role can still broaden access more than a world-readable env file would.

Comparison Table

Which to Choose

Local development and prototypes

Use environment variables. A .env file keeps the key out of source control and matches production shape closely enough.

Single production service, low compliance burden

Environment variables sourced from a sealed secret store (Kubernetes Secrets with encryption at rest, Docker secrets, or SOPS-encrypted files) are sufficient. You avoid per-call fees and network dependency.

Multi-service platform or regulated workload

Adopt a secret manager. The ability to rotate a leaked key by updating one version, and to audit which role read it, is worth the SDK boilerplate. Use an operator to project values as env vars so application code stays unchanged.

High-throughput LLM inference or multi-provider routing

When you front models through a gateway that honors client routing directives and forwards provider cache-control hints—n4n.ai does this for its OpenAI-compatible endpoint across 240+ models—you may keep provider keys in a vault and resolve them per route. Cache the fetched key in memory to avoid adding latency on every token stream, and rely on the gateway’s automatic fallback when a provider is degraded. The secret manager becomes the source of truth; the gateway becomes the consumer.

Serverless and ephemeral compute

Functions scale to thousands of cold starts per minute. Fetch the secret once in the init phase, not per invocation, or you will hit API quotas fast. Environment variables injected by the platform at deploy time are often the simpler path if your CI can write them securely.

The core trade is static simplicity versus dynamic control. Environment variables vs secret managers api keys is not a moral question; it is a scope question. Pick the smallest tool that survives your next incident.