Docker images for reproducible AI agent deployments

Docker AI agent deployment fails when the image built on your laptop behaves differently in production because a transitive dependency shifted or a model endpoint changed. Reproducibility means the container encloses code, runtime, and model interaction contracts, not just a Python script.

1. Pin the base image by digest

Float tags like python:3.12 resolve to different layers week to week. A Docker AI agent deployment must start from a known filesystem. Pull the digest from your registry and lock it.

FROM python:3.12.3-slim@sha256:8c9a3f... # real digest from `docker pull` output

Tradeoff: slim images exclude build toolchains. If you need gcc to compile a wheel, do it in a separate stage and discard the compiler. Never ship build-essential in the runtime image.

2. Isolate dependencies with multi-stage builds

Compile wheels in a builder stage. The final image receives only the built artifacts, keeping the layer hash stable and the CVE surface small.

FROM python:3.12.3-slim@sha256:8c9a3f... AS builder
COPY requirements.txt .
RUN pip wheel --no-cache-dir --wheel-dir /wheels -r requirements.txt

FROM python:3.12.3-slim@sha256:8c9a3f...
COPY --from=builder /wheels /wheels
RUN pip install --no-cache-dir /wheels/* \
    && rm -rf /wheels

This pattern removes network access requirements at deploy time. The agent container installs nothing on boot.

3. Freeze dependencies with hashed locks

Generate a deterministic resolution. uv or pip-tools emit hashes that fail loudly if a package changes upstream.

uv pip compile requirements.in -o requirements.txt --generate-hashes

Commit requirements.txt. A Docker AI agent deployment pipeline that runs pip install against unpinned indexes is one PyPI compromise away from silent behavior change.

System libraries count as dependencies

If your agent parses PDFs or images, you pull shared objects. Pin those too.

RUN apt-get update && apt-get install -y --no-install-recommends \
    libmagic1=1:5.44-3 \
    poppler-utils=22.12.0-2 \
    && rm -rf /var/lib/apt/lists/*

Missing pins here cause the same drift as a Python package shift.

4. Externalize model routing and secrets

Never embed API keys in the image. Pass them at runtime through Docker secrets or an init process that reads from a vault. Model selection belongs in environment, not code.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LLM_GATEWAY_URL"],
    api_key=os.environ["LLM_API_KEY"],
)
MODEL = os.environ.get("AGENT_MODEL", "gpt-4o-mini")

Using a gateway that exposes one OpenAI-compatible endpoint for many models lets you swap providers without rebuilding. n4n.ai does this with 240+ models and automatic fallback, so the container stays identical across provider outages.

Pitfall: env var defaults hide misconfiguration

If AGENT_MODEL defaults to a model your gateway doesn’t serve, the container boots but fails on first call. Validate env at startup:

required = ["LLM_GATEWAY_URL", "LLM_API_KEY"]
missing = [k for k in required if not os.environ.get(k)]
if missing:
    raise SystemExit(f"Missing env: {missing}")

5. Make agent behavior deterministic where you control it

You cannot freeze LLM weights inside the container, but you own temperature, seed, and tool schemas. Set them explicitly.

response = client.chat.completions.create(
    model=MODEL,
    temperature=0.0,
    seed=42,
    messages=messages,
    tools=tool_schemas,
)

Cache-control hints forwarded by the gateway reduce latency and cost. Send them when your client allows:

client.chat.completions.create(
    model=MODEL,
    messages=messages,
    extra_headers={"cache-control": "max-age=300"},
)

Tradeoff: reproducibility vs. exploration

Pinning temperature=0 helps integration tests. Production agents often need variability. Gate these via AGENT_ENV=prod|test rather than separate images.

6. Build with BuildKit and exclude noise

Enable BuildKit. Use .dockerignore to keep .git, venv, and __pycache__ out of the build context. A stray .pyc changes the layer hash and breaks reproducibility claims.

DOCKER_BUILDKIT=1 docker build --build-arg BUILD_DATE=2024-01-01 -t agent:1.0.0 .

Avoid RUN date or RUN git clone at build time. Those embed non-deterministic values into the image.

7. Validate the image as a black box

Spin the container in CI against a mock LLM. Assert the agent calls tools correctly and respects the contract.

# docker-compose.test.yml
services:
  agent:
    image: agent:1.0.0
    environment:
      LLM_GATEWAY_URL: http://mock:8080
      LLM_API_KEY: test
      AGENT_MODEL: mock-model
    ports:
      - "8000:8000"
  mock:
    image: kennethreitz/httpbin

Run a pytest session that POSTs to the agent’s /run endpoint and checks the response schema. If the container logs to stdout, capture it and grep for expected spans.

Common pitfall: locale and timezone

Missing LANG breaks Unicode logging. Set it explicitly.

ENV TZ=UTC
ENV LANG=C.UTF-8

Without TZ=UTC, timestamped traces differ across hosts and corrupt log correlation.

8. Run as non-root and read-only

Create a user. Drop privileges before the entrypoint.

RUN useradd -m appuser
USER appuser

Launch with a read-only root filesystem. Mount writable space only for explicit caches.

docker run --read-only --user appuser -e LLM_API_KEY agent:1.0.0

If the agent writes to /tmp, add --tmpfs /tmp. A writable root fs invites state drift between restarts.

9. Tag by content, not by time

A timestamp tag like agent:20240101 hides what’s inside. Use the git SHA plus a hash of the lockfile.

TAG=$(git rev-parse HEAD)-$(sha256sum requirements.txt | cut -c1-8)
docker tag agent:1.0.0 agent:${TAG}

A Docker AI agent deployment audit starts by mapping a running container ID to that tag, then to the exact source tree.

10. Capture provenance with an SBOM

Generate a software bill of materials during build. syft produces a JSON you can store alongside the image.

syft packages docker:agent:1.0.0 -o json > sbom.json

When a CVE hits a transitive package, you query the SBOM instead of pulling the image apart. This is cheaper than rescanning every running host.

11. Document the runtime contract inside the image

Ship a CONTRACT.md at /app/CONTRACT.md. List required env vars, exposed ports, and expected model capabilities (function calling, JSON mode). docker inspect shows ports but not semantics.

COPY CONTRACT.md /app/CONTRACT.md

On-call engineers should understand what the container promises without reading the Dockerfile.

12. Treat the container as the unit of promotion

Promote the same image from staging to production. Do not rebuild per environment. Environment differences enter through mounted config and secrets only.

A Docker AI agent deployment that rebuilds on each environment inherits the risk of pulling a newer patch of a base image or a different wheel. Build once, tag by content, promote the artifact.

Common pitfalls summary

• Baking secrets: they survive in layer history even if later removed.
• Using latest for any parent image or tool.
• Installing dev packages in the final stage.
• Relying on host TZ or LANG.
• Skipping black-box tests against a mock model.

Reproducibility is not a feature you add at the end. It is the sum of every pin, every excluded file, and every externalized variable above. Get those right and the agent runs the same on your laptop, in CI, and at 3 a.m. in production.