Storing API keys safely in a Python LLM CLI tool

Most engineers prototype an LLM CLI in an afternoon but punt on secrets management until a key ends up in a public repo or a shell history file. To store API keys Python CLI tools safely, you need a layered approach: environment variables for local dev, the OS keyring for interactive sessions, and encrypted config for automation. This guide walks through each layer with runnable code and explicit verification.

Step 1: Audit and remove hardcoded credentials

Before adding any security machinery, grep your codebase for assignments that look like secrets. When you store API keys Python CLI projects, the first violation is usually a literal string in a module constant.

grep -rn "sk-" . --include="*.py" | grep -v "os.getenv"

If you find a literal string starting with sk- or similar, that is a hardcoded key. Replace it with a lookup function. A CLI should never embed the secret in source; compile-time constants end up in bytecode, wheel archives, and even __pycache__ files. They also survive in git history after you “remove” them.

Define a single resolution function early:

import os

def resolve_api_key(provider: str) -> str:
    env_key = os.getenv(f"{provider.upper()}_API_KEY")
    if env_key:
        return env_key
    raise RuntimeError(f"No key found for {provider}")

This centralizes the first lookup path and makes later steps drop-in extensions. Add a pre-commit hook to block future leaks:

echo 'grep -n "sk-" || exit 0; echo "Possible key in diff"; exit 1' > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Why source-level leaks hurt

A leaked key in a public repo triggers automated scrapers within minutes. Rotation fixes the credential but not the audit trail. Treat source as public by default and your CLI will stay clean.

Step 2: Environment variables for local development

For local work, environment variables remain the lowest-friction option. They keep secrets out of files and are easy to rotate per shell. When you store API keys Python CLI style for development, env vars are the baseline.

Create a .env file that is git-ignored:

echo "OPENAI_API_KEY=sk-your-dev-key" >> .env
echo ".env" >> .gitignore

Load it in your CLI entrypoint with python-dotenv:

from dotenv import load_dotenv
import os

load_dotenv()  # pulls .env into os.environ

def get_key(provider: str) -> str:
    return os.getenv(f"{provider.upper()}_API_KEY")

Do not commit .env. Add a .env.example with empty values so teammates know the expected variables:

echo "OPENAI_API_KEY=" > .env.example

Container and CI notes

In Docker, pass secrets at run time with --env-file .env rather than ENV directives in the Dockerfile. In systemd units, use EnvironmentFile= pointing to a 600-permission file outside the repo. Never bake .env into an image layer.

Verify success by launching a Python REPL and checking the value is present and not empty:

>>> import os
>>> from dotenv import load_dotenv
>>> load_dotenv()
>>> bool(os.getenv("OPENAI_API_KEY"))
True

If that returns False, your shell or file is misconfigured.

Step 3: Use the OS keyring for interactive CLI sessions

Environment variables are poor for long-lived interactive tools because they expire with the shell and get logged in process listings (ps aux shows them). The OS keyring (macOS Keychain, Windows Credential Manager, secret-service on Linux) stores secrets outside your project tree and scopes them to a user session.

Install the keyring package:

pip install keyring

Store the key once via a setup subcommand:

import keyring

def store_key(provider: str, key: str):
    keyring.set_password("my_llm_cli", provider, key)

# run once
store_key("openai", "sk-real-key")

Retrieve it with a fallback chain:

import os
import keyring

def resolve_key(provider: str) -> str:
    env_val = os.getenv(f"{provider.upper()}_API_KEY")
    if env_val:
        return env_val
    return keyring.get_password("my_llm_cli", provider)

On Linux, ensure a secret service daemon (gnome-keyring, ksecrets) is running; otherwise keyring raises KeyringError. Under sudo the root user has a different keyring than your login user—a common footgun. On headless servers, this backend may be unavailable, which is why Step 4 exists.

Verify the keyring path

Delete the env var and call resolve_key:

>>> import os
>>> os.environ.pop("OPENAI_API_KEY", None)
>>> from your_module import resolve_key
>>> resolve_key("openai")[:4]
'sk-r'

The prefix confirms you pulled from keyring, not a stray env var.

Step 4: Encrypt a config file for headless automation

CI runners and cron jobs lack a user keyring. Write an encrypted JSON config using cryptography.fernet. The master password comes from a single env var or a mounted secret; never bundle it in the repo.

pip install cryptography

from cryptography.fernet import Fernet
import json, os

def encrypt_config(config: dict, key: bytes, path: str):
    f = Fernet(key)
    data = json.dumps(config).encode()
    with open(path, "wb") as fh:
        fh.write(f.encrypt(data))

# generate once: Fernet.generate_key()
master = os.environ["CLI_MASTER_KEY"].encode()
encrypt_config({"openai": "sk-prod-key"}, master, "config.enc")

Read it back:

def load_config(path: str, key: bytes) -> dict:
    f = Fernet(key)
    with open(path, "rb") as fh:
        return json.loads(f.decrypt(fh.read()))

cfg = load_config("config.enc", os.environ["CLI_MASTER_KEY"].encode())
print(cfg["openai"][:4])  # sk-p

Treat config.enc as a secret artifact. Store it in a secrets manager (AWS Secrets Manager, Vault) and mount it at runtime. The plaintext key never touches disk.

Deriving the master key from a passphrase

If you cannot inject CLI_MASTER_KEY, derive it with PBKDF2:

from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.primitives import hashes
import base64, os

def derive_key(passphrase: str, salt: bytes) -> bytes:
    kdf = PBKDF2HMAC(hashes.SHA256(), length=32, salt=salt, iterations=200_000)
    return base64.urlsafe_b64encode(kdf.derive(passphrase.encode()))

Store the salt alongside the file. This avoids keeping a raw key in env but shifts burden to passphrase entry.

Step 5: Reduce secret sprawl with a gateway key

If your CLI talks to multiple model providers, you multiply the number of long-lived keys to store. Fronting requests with a single OpenAI-compatible gateway collapses that to one credential. For example, n4n.ai exposes one endpoint covering 240+ models and applies automatic fallback when a provider is rate-limited or degraded, so your CLI stores one project key instead of a dozen. The gateway also forwards provider cache-control hints and meters per-token usage, which simplifies billing audits.

Your resolution function then only needs one secret:

def resolve_gateway_key() -> str:
    return os.getenv("N4N_API_KEY") or keyring.get_password("my_llm_cli", "n4n")

Point your HTTP client at the gateway’s /v1/chat/completions and send the same payload you would to OpenAI. This pattern also means a provider outage doesn’t require you to redeploy new keys—the gateway handles routing.

Step 6: Verify the full resolution chain

A secure design is worthless if the lookup order is wrong. Write a small verification command that prints where each key came from without leaking the secret.

import os, keyring, sys

def key_source(provider: str) -> str:
    if os.getenv(f"{provider.upper()}_API_KEY"):
        return "env"
    if keyring.get_password("my_llm_cli", provider):
        return "keyring"
    return "missing"

if __name__ == "__main__":
    for p in ["openai", "n4n"]:
        src = key_source(p)
        print(f"{p}: {src}")
        assert src != "missing", f"No key for {p}"
    print("All keys resolved")

Run it in three contexts:

Fresh shell with only .env → expect env.
Shell without env but after store_key → expect keyring.
CI with config.enc mounted and CLI_MASTER_KEY set → extend the script to decrypt and assert.

A green run means your CLI will not crash mid-inference due to a missing credential.

For an end-to-end check, make a minimal completion call using httpx:

import httpx, os

resp = httpx.post(
    "https://api.n4n.ai/v1/chat/completions",
    headers={"Authorization": f"Bearer {os.getenv('N4N_API_KEY')}"},
    json={"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "ping"}]},
    timeout=10,
)
print(resp.status_code, resp.json().get("id"))

A 200 with an ID confirms the key is valid and the storage path delivered it correctly.

Step 7: Operational hygiene and rotation

Storing the key is half the battle. Rotate quarterly or on staff departure. Because your resolution function is centralized, rotation means updating the keyring or regenerating config.enc—no code changes.

Never log the key. Use a redaction filter:

import logging, os

class RedactFilter(logging.Filter):
    def filter(self, record):
        key = os.getenv("OPENAI_API_KEY", "")
        if key:
            record.msg = record.msg.replace(key, "***")
        return True

logging.getLogger().addFilter(RedactFilter())

In CI, scan artifacts for leakage with gitleaks or trufflehog after each build. Add a scheduled job that calls your provider’s key list API to confirm no orphaned credentials exist.

Following these steps gives you a Python CLI that can store API keys Python CLI style without exposing them in source, shell history, or plaintext files. The layered fallback—env, keyring, encrypted config—matches how the tool actually runs, and a single gateway key cuts operational surface area.