The cold-start tax
Scale-to-zero is a great deal: idle apps cost nothing. The bill comes due on the first request after idle, when the platform has to bring a pod from zero to serving. That request waits through the whole cold start while warm requests are fast. The goal of this post is to make that cold path as short as possible.
First, you can't optimize what you don't decompose. A cold start is a sum of phases:
Total cold start =
scheduling (find a node, admit the pod)
+ image pull (download container image, if not cached)
+ container start (create the sandbox, mount, init)
+ app boot (your process starts, opens connections, warms caches)
+ first-request work (JIT, lazy init, connection establishment)Each phase has different levers. Let's go through them in roughly the order of biggest typical wins.
1. Shrink your image (often the biggest lever)
Image pull is frequently the dominant cost on a cold node. A 1.2 GB image pulls far slower than a 60 MB one. Attack image size hard:
- Multi-stage builds. Compile in a fat builder stage, copy only the artifact into a tiny runtime stage.
- Minimal base images.
node:20-slim,python:3.12-slim,distroless, oralpine(mind glibc/musl differences) over full distros. - Don't ship build tooling. No compilers, no dev dependencies in the final image.
# Multi-stage: builder is huge, runtime is tiny
FROM node:20 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm prune --omit=dev
FROM node:20-slim AS runtime
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
CMD ["node", "dist/server.js"]Smaller images also pull faster on *every* deploy, not just cold starts — a compounding win.
2. Keep image layers cached on nodes
If a node has already pulled your image layers, the pull phase nearly vanishes. Techniques:
- Stable base layers. Put rarely-changing layers (base image, dependencies) early in the Dockerfile so they're cached and shared; put your fast-changing source last.
- Registry locality. Pull from a registry close to your nodes (same region/cloud). PandaStack ships images to Google Artifact Registry co-located with its GKE nodes, so pulls stay in-region.
- Image streaming / lazy pulling. Some platforms stream image content on demand so the container starts before the full image is local.
3. Make your app boot fast
Once the container starts, your process has to come up. Common boot-time sinks:
- Eager initialization. Apps that connect to every dependency, run migrations, warm large caches, or load big models at startup pay all of it on the cold path. Defer non-critical init until after the server is listening.
- Heavy frameworks/runtimes. A minimal HTTP server boots in tens of milliseconds; a heavyweight framework with lots of reflection/DI can take seconds. Choose proportionally to the workload.
- Connection pools. Establishing DB connections is slow. Open a *small* pool at boot and grow lazily, rather than blocking startup on a large pool.
A good rule: start listening as fast as possible, then warm in the background. Your readiness probe gates traffic until you're actually ready, but you don't want the readiness gate held by work that could happen after the first request.
4. Tune the autoscaler, not just the app
Sometimes the fix is to cold-start *less often* rather than faster:
- Cooldown period. If your app scales to zero after 60 seconds of idle but gets traffic every 90 seconds, you cold-start constantly. Lengthen the idle window so genuinely-active apps stay warm.
- Minimum warm replica for production. The cleanest fix for user-facing production traffic is simply *not* scaling to zero — keep
minReplicas: 1. Reserve scale-to-zero for dev/preview/intermittent workloads where the trade is worth it. - Pre-warming on a schedule. If traffic is predictable (business hours), a cronjob that pings the app just before peak keeps a pod warm when it matters.
5. Right-size the sandbox
Stronger isolation costs a bit of start time. gVisor sandboxes and microVMs add some startup overhead versus bare runc. That's usually a worthwhile security trade, but it's a knob: latency-critical paths may warrant a lighter isolation tier, while untrusted/multi-tenant workloads keep the stronger sandbox.
A phase-by-phase cheat sheet
| Phase | Biggest lever | Quick win |
|---|---|---|
| Scheduling | Node availability/warm pool | Pre-warm before predictable peaks |
| Image pull | Image size + cache locality | Multi-stage build, slim base, in-region registry |
| Container start | Isolation tier | Right-size sandbox for the workload |
| App boot | Lazy init | Listen first, warm in background |
| First request | Connection/JIT warm-up | Small lazy pools, warm critical paths |
How this plays out on PandaStack
PandaStack's free tier uses KEDA scale-to-zero on spot nodes inside a gVisor sandbox — which is what makes a generous free tier sustainable, and it means free-tier apps do cold-start (we say so plainly in our limits). The platform already does the heavy lifting on the infra side: in-region Google Artifact Registry pulls and rootless-built images. Your job is the app side — keep the image small and the boot lazy. For production traffic where cold starts aren't acceptable, run on a tier that keeps a warm replica rather than scaling to zero. The same techniques (slim images, fast boot) make *every* deploy and *every* scale-up faster, so they're worth doing regardless of tier.
Bottom line
Cold starts aren't a single number you tweak — they're a pipeline you optimize phase by phase. The two highest-leverage moves for most apps: shrink the image (faster pulls) and boot lazily (listen first, warm later). Tune autoscaler cooldowns so active apps stay warm, and keep production user-facing traffic on a warm replica. Do that and scale-to-zero gives you the cost savings without making your users feel them.
References
- [Docker — Multi-stage builds](https://docs.docker.com/build/building/multi-stage/)
- [Google — Distroless container images](https://github.com/GoogleContainerTools/distroless)
- [Kubernetes — Configure liveness/readiness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/)
- [KEDA — Scaling deployments (cooldownPeriod)](https://keda.sh/docs/latest/concepts/scaling-deployments/)
Want scale-to-zero with in-region image pulls handled for you? PandaStack's free tier does the infra side automatically — bring a slim image and go: [dashboard.pandastack.io](https://dashboard.pandastack.io)