Running Is Not Ready: Operating Containerized Services

One of the most reliable lessons from operating containerized services is simple: running is not ready. A process can be alive, a port can be open, and yet the service may not be able to handle real traffic. Treat startup, readiness, and liveness as separate concerns, verify each one deliberately, and design rollouts with recovery in mind.

This article uses a small public demo, aceu-trust-demo, as a running example. It uses a multi-stage image, a non-root user, a health endpoint, and a container deployment behind external HTTPS ingress. Its ACE-U output is illustrative and based on synthetic evidence; it is not a validated assessment of real people. The operational lessons apply more broadly.

The “green” deployment that still fails

A deployment reports “running” and the dashboard looks healthy, but requests time out. A dependency is still warming up. The process is alive, so the liveness signal stays green; the service is simply not ready to serve traffic yet. The failure began when one signal was expected to answer two different questions.

Give each signal one job

Startup: has initialization finished?

Startup covers work such as loading configuration, opening required connections, or priming a cache. Slow startup should not be mistaken for a dead process.

Readiness: can this instance take traffic now?

Readiness is a promise to callers. Keep it false until the dependencies required for the service’s critical path are available. A TCP connection alone may not be enough if the application depends on a loaded index or credentials.

Liveness: is the process stuck in a state that needs a restart?

Liveness should detect an unrecoverable condition, such as a deadlock. It should not fail merely because a downstream system is briefly unavailable: repeatedly restarting healthy processes can amplify an outage.

Docker’s HEALTHCHECK records a container health state. It does not, by itself, define how a particular load balancer or orchestrator routes traffic. In Kubernetes, configure startup, readiness, and liveness probes for their distinct purposes.

A bounded Docker health check

If Python is already part of the runtime image, a tiny local endpoint check avoids adding a separate curl package just for health status:

HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD python -c 'import urllib.request; urllib.request.urlopen("http://localhost:8000/health", timeout=2)' || exit 1

Keep this endpoint cheap and local. Put stricter dependency checks in a separate readiness endpoint, then configure your platform to use that endpoint to gate traffic.

Roll out in stages and predefine rollback

Do not move all traffic at once. Start with a small cohort, hold it under real load for a defined soak period, then expand. Write the rollback condition before the release—for example, a measured latency threshold sustained for a defined duration, or an error rate that crosses a set limit. The thresholds should come from the service’s baseline and risk profile, not be guessed after the rollout starts.

Shut down gracefully

When a platform scales down an idle container, the process may receive SIGTERM. Stop accepting new work, let in-flight requests finish within a deadline, close outbound clients, flush metrics, and exit. Treat SIGKILL as the forced fallback. Without a drain period, a routine scale-down can become partial writes, dangling sessions, or noisy retries.

Make the dependency chain visible

Under pressure, teams often blame the topmost service. Keep a runbook that follows dependencies below the application: runtime, network, storage, cache or index, and external services. Document how each layer fails and how to check it without adding load. Infrastructure problems such as CPU starvation, memory pressure, or network saturation can surface as application timeouts and backpressure.

A green health endpoint can also miss a heavy or rarely used path. Pair alerts with lightweight, non-sensitive transition logs such as ready=false reason=index_warming and ready=true checks=passed. A weekly on-call handoff—what broke, the likely cause, and which mitigation helped—turns recurring cold starts and slow dependency warm-ups into work the team can fix.

Trade-offs to make explicit

• Strict readiness vs. first-green speed: better gating can delay traffic, while reducing user-facing failures.
• Longer soak vs. delivery pace: more observation lowers rollout risk but delays full exposure.
• Graceful drain vs. implementation effort: draining takes work; skipping it risks incomplete operations.
• Frequent liveness checks vs. flapping: overly sensitive checks can restart a service during resource spikes and worsen an incident.

Try it in your service

1. Add a separate /ready endpoint that checks the critical dependency before returning success.
2. Write down a staged rollout, soak period, and rollback trigger before your next release.
3. Handle SIGTERM by draining requests and closing resources cleanly.
4. Extend your runbook with infrastructure checks and first suspects for latency spikes.

Closing thought

Separating startup, readiness, and liveness reduces confusion during incidents and helps limit their blast radius. The demo illustrates the distinction with a local health endpoint and a reversible deployment path. Its ACE-U dimensions—Authenticity, Credibility, Empathy, and Uniqueness—remain conceptual in this context, not a judgment of real people.