How to Configure Health Checks for Your Deployments

Why health checks exist

When you deploy an app, the orchestrator needs to answer two questions: "Is this instance alive?" and "Is this instance ready to receive traffic?" Without health checks, a load balancer happily routes requests to a process that's crashed, deadlocked, or still booting, and your users get errors. Health checks are the contract that lets the platform make smart routing and restart decisions.

The three kinds of probes

Kubernetes (which underpins most modern platforms) defines three probe types, and understanding the difference is the whole game:

The most important distinction: liveness failures restart your container; readiness failures just stop traffic. Confusing the two is the source of most health-check disasters.

Designing good health endpoints

Liveness should be cheap and local

A liveness probe should answer "is my event loop responsive?" and nothing more. It must not check the database, downstream APIs, or anything external. Why? If your database has a brief hiccup and your liveness probe checks the DB, every instance fails liveness simultaneously, gets killed simultaneously, and you turn a minor blip into a full outage, a restart storm.

// Good liveness: trivial, no external dependencies
app.get('/healthz', (req, res) => res.status(200).send('ok'));

Readiness can check dependencies

Readiness is where you check whether this instance can actually do its job: is the database reachable, is the cache connected, has the app finished warming up?

// Readiness: verify critical dependencies
app.get('/readyz', async (req, res) => {
  try {
    await db.query('SELECT 1');         // DB reachable
    if (!cache.isConnected()) throw new Error('cache down');
    res.status(200).send('ready');
  } catch (e) {
    res.status(503).send('not ready');  // pulled from LB, not killed
  }
});

When readiness fails, the instance is pulled from rotation but kept alive, so when the dependency recovers, it rejoins automatically. No restart needed.

Startup probes for slow boots

Apps that load large models, run migrations, or warm caches can take a while to start. A startup probe gives them grace before liveness kicks in, so they aren't killed mid-boot.

Kubernetes probe configuration

livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10
  failureThreshold: 3      # 3 failures (~30s) before restart
readinessProbe:
  httpGet:
    path: /readyz
    port: 8080
  periodSeconds: 5
  failureThreshold: 2
startupProbe:
  httpGet:
    path: /healthz
    port: 8080
  failureThreshold: 30     # up to 30 * 10s = 5 min to start
  periodSeconds: 10

Common mistakes

• Checking the DB in liveness → restart storms during DB blips. Don't.
• initialDelaySeconds too short → app killed before it finishes booting. Use a startup probe instead of guessing.
• failureThreshold too aggressive → transient network blips trigger restarts. Give a couple of retries.
• Readiness that never recovers → make sure a failed dependency, once healthy, flips readiness back to 200.
• Same endpoint for liveness and readiness → you lose the ability to pull-without-killing. Use separate endpoints.

Health checks on a managed platform

On platforms that run your apps on Kubernetes, health checks are configured for you with sensible defaults, but you still own the *content* of the endpoints. PandaStack deploys apps via Helm on multi-region GKE, so your container benefits from Kubernetes-style probes during rolling deploys, an instance only receives traffic once it reports ready, which is what makes zero-downtime deploys and clean rollbacks possible.

The practical implication: even on a managed platform, expose a cheap /healthz (liveness) and a dependency-aware /readyz (readiness) in your app. That's the part the platform can't write for you, and it's what makes deploys safe.

# FastAPI example with separate endpoints
from fastapi import FastAPI, Response
app = FastAPI()

@app.get('/healthz')
def live():
    return {'status': 'ok'}  # no external calls

@app.get('/readyz')
def ready(response: Response):
    if not db_ok():
        response.status_code = 503
        return {'status': 'not ready'}
    return {'status': 'ready'}

Testing your probes

Before trusting them in production:

1. 1Hit /healthz and /readyz directly with curl, confirm status codes.
2. 2Kill the database and confirm readiness goes 503 but liveness stays 200.
3. 3Confirm that when the DB returns, readiness recovers without a restart.
4. 4Simulate a slow boot and confirm the startup probe holds off liveness.

Conclusion

Good health checks come down to one rule: liveness is local and cheap; readiness checks dependencies and is recoverable. Get that split right and your platform will route around problems instead of amplifying them. Get it wrong and a five-second database blip becomes a cluster-wide restart storm.

PandaStack runs your apps on GKE with rolling deploys gated on readiness, so a clean /readyz endpoint gives you zero-downtime deploys out of the box. Try it on the free tier at https://dashboard.pandastack.io.

References

• Kubernetes liveness, readiness, and startup probes: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
• Kubernetes probe configuration reference: https://kubernetes.io/docs/concepts/configuration/liveness-readiness-startup-probes/
• Google SRE book, on cascading failures: https://sre.google/sre-book/addressing-cascading-failures/
• The Twelve-Factor App, disposability: https://12factor.net/disposability