Keys Don’t Belong in the Image: Secret Management for Credential Signing

Signing keys are crown jewels in identity services. If they leak, attackers can mint forged credentials that look real. The easiest way to leak a key is to bake it into a container image or configuration file. Load signing keys at runtime from a managed secret store; keep private key material out of source files, build arguments, and image layers.

The failure mode

A contributor tests locally with a .env file containing a signing key. A sloppy Dockerfile copies everything under the project directory, including .env. CI builds the image; the key ends up in a layer. The image is pushed to a registry. Even if you rotate later, the leak is permanent in history.

What I learned

Secrets never live in source or image layers

Keep signing keys out of your repository and artifact layers. Use a managed secret store pattern to provide keys at runtime via secure mounts or platform-managed environment variables that never touch the image.

.dockerignore is a security control

Treat .dockerignore as part of your security posture. Explicitly ignore .env, credential files, test fixtures containing secrets, and local caches. Keep the build context as small as possible.

Runtime retrieval, not build-time embedding

Load keys at runtime from the secret store client library or a platform-provided mount. Fail readiness until the key loads. If the secret is missing, return clear errors and refuse to sign. Do not “fall back” to dummy keys in production—reserve test keys for explicit non-prod builds.

Rotation: short lifetimes and multiple active keys

Signing keys should rotate regularly. The verifier should accept signatures from a small set of current keys, and the issuer should roll keys with overlap: publish new key, start signing with it, keep the old key valid for verification until the overlap window expires.

Audit and scanning: prevent accidental bake-ins

Add CI checks that scan images for secret patterns. Simple rules catch common mistakes (e.g., files named .env, PEM key headers). Block the build if found. Complement this with code scanning that flags dangerous COPY patterns.

Separate test and prod key handling

For demos, use explicitly marked test keys with sandboxed scopes. Never reuse test keys in environments accessible outside controlled demos. Label logs and status endpoints so it’s obvious which environment your verifier belongs to.

Minimize key surface: hardware-backed or managed services

Where possible, use hardware-backed key stores or provider-managed keys with restricted export. Keep application access tightly scoped to “sign” and “read public key.” The app should never be able to exfiltrate private key material.

Illustrative runtime secret loading

def load_signing_key(secret_store, key_id: str) -> bytes:
    key = secret_store.read(key_id)
    if not key:
        raise RuntimeError("signing key unavailable")
    return key

# Load keys from a managed store at runtime. Never copy key material
# into source, build arguments, container layers, or sample configuration.
# Keep readiness false until the signing dependency is ready.

Example .dockerignore

.env
*.pem
__pycache__/
*.pyc
.git/
node_modules/
tests/fixtures/secrets/

Example Dockerfile pattern

FROM python:3.12-slim AS runtime
WORKDIR /app
COPY pyproject.toml poetry.lock ./
RUN pip install --no-cache-dir poetry && poetry install --no-root
COPY --chown=10001:10001 app ./app
USER 10001
EXPOSE 8000

Trade-offs and failure modes

Trade-offs

Runtime dependency vs build independence: Secret store clients add runtime requirements. The gain is eliminating baked-in risk.

Rotation complexity vs safety: Overlapping keys complicate issuers and verifiers. The gain is smooth migration.

Strict readiness vs availability: Refusing to start without keys prevents undefined behavior. Availability dips are better than silent mis-signing.

Failure modes

“COPY .”: Blindly copying the entire source tree into the image. Fix with .dockerignore and explicit COPY.

“Fallback key”: Using a dummy key in production when secret load fails. Fix with hard failures and improved observability.

“Exportable private keys”: Allowing the app to export private keys. Fix with hardware-backed or provider-managed keys.

Try it yourself

Add .dockerignore entries for secrets and verify your build context (run docker build --no-cache and inspect logs).

Use strict readiness that fails until keys load from a managed store. Test rotation: accept signatures from two keys during an overlap window.

Add CI scans that fail on secret-like files in images. Track and eliminate false positives carefully.