Cold Starts Deep Dive: Causes and How to Reduce Them

A cold start is the latency penalty a request pays when there's no warm instance ready to serve it — the platform must spin one up first. Scale-to-zero saves money by running nothing when idle, but the first request after idling waits for that spin-up. This deep dive dissects exactly where cold-start time goes and how to reduce each component.

What "cold" means

When your app scales to zero (or a serverless function has no warm container), there's no process waiting. A request arriving in that state triggers the full spin-up sequence before any of your code runs. "Warm" means an instance is already up; "cold" means one must be created. The cold-start cost is the difference.

Anatomy of a cold start

A cold start is not one thing — it's a chain of phases, each contributing latency:

1. Scheduling     → find/allocate a node for the pod
2. Image pull     → download the container image (if not cached on node)
3. Container init  → start the container, sandbox setup
4. Runtime boot   → start the language runtime (Node/JVM/Python)
5. App init        → your startup code (DB connections, config, warmup)
6. First request   → finally serve the request

The total cold start is the sum. Optimizing means attacking whichever phases dominate *your* app — and they vary a lot by stack.

Phase 1: Scheduling

The orchestrator must find a node with room for the pod. On a system using spot/preemptible nodes (cheap, but capacity isn't pre-reserved), scheduling can include waiting for a node. PandaStack's free tier runs on spot nodes with KEDA scale-to-zero — great economics, but scheduling is a real phase.

Reduce it by: keeping a warm-node buffer (paid platforms often do), or accepting it as the cost of true scale-to-zero on free/spot capacity.

Phase 2: Image pull

If the target node doesn't already have your image cached, it must pull it from the registry. A 2 GB image pulls far slower than a 50 MB one. This is frequently the single biggest cold-start contributor.

Reduce it by:

• Shrinking the image. Use slim/distroless bases and multi-stage builds.
• Removing build-only deps from the final image.
• Layer caching so common layers are already on nodes.

# Multi-stage: ship only the runtime artifacts
FROM node:20 AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-slim   # much smaller final image
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
CMD ["node", "dist/server.js"]

Phase 3: Container init / sandbox

Starting the container and any isolation layer adds time. PandaStack's free tier wraps apps in a gVisor sandbox for security; sandboxing adds a small, generally worthwhile overhead. This phase is mostly fixed, but a lean image and a fast entrypoint help.

Phase 4: Runtime boot

Different runtimes boot at very different speeds:

Reduce it by: trimming dependencies (every import/require at top level runs on boot), lazy-loading heavy modules, and for the JVM exploring AOT/CDS options.

Phase 5: App initialization — where you have the most control

This is your code, so it's where you can win the most. Common offenders:

• Synchronous, blocking startup: opening DB connections, reading large config, warming caches before the server can accept requests.
• Eager work that could be lazy: precomputing things you might not need.

Reduce it by:

• Defer non-essential work. Start the HTTP server first; warm caches in the background.
• Lazy-connect. Open the DB connection on first use, not at boot — or use a fast connection pool init.
• Fail readiness, not liveness, during warmup so traffic waits for genuine readiness without restarting the pod.

// Start serving fast; warm in the background
const app = createServer();
app.listen(PORT, () => console.log('listening'));

// Non-blocking warmup
setImmediate(async () => {
  await db.connect();
  await cache.warm();
});

Measuring where the time goes

Don't optimize blind. Instrument the phases:

• Log a timestamp at process start and again when the server is ready → that's your runtime + app-init time.
• Compare image-pull-cached vs uncached cold starts to isolate pull time.
• Use the platform's server-side metrics to see end-to-end first-request latency after idle.

Optimizing the wrong phase is wasted effort. A 1.5 GB image won't be saved by shaving 50 ms off app init.

The economic trade-off

Scale-to-zero exists to save money: an idle app costs nothing. The price is the first-request latency after idle. The right call depends on the workload:

• Dev, hobby, preview, low-traffic: scale-to-zero is ideal — who cares about one slow request after hours of idle? This is exactly why PandaStack's free tier uses it (spot nodes + KEDA + gVisor).
• Latency-critical production: keep a minimum of one warm instance (a paid, non-preemptible tier), trading a little cost for consistent latency.

Be honest about which you have. Don't put a user-facing checkout flow on scale-to-zero and then complain about cold starts.

Quick-win checklist

• [ ] Shrink the image (slim/distroless, multi-stage)
• [ ] Remove build-only deps from the final image
• [ ] Trim top-level imports; lazy-load heavy modules
• [ ] Start the server before warming caches/DB
• [ ] Use readiness probes correctly
• [ ] Keep a warm instance for latency-critical paths
• [ ] Measure per-phase before optimizing

References

• [AWS: Operating Lambda — cold starts](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/)
• [Kubernetes pod lifecycle and probes](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/)
• [Distroless container images](https://github.com/GoogleContainerTools/distroless)
• [KEDA scaling concepts](https://keda.sh/docs/latest/concepts/)
• [Docker multi-stage builds](https://docs.docker.com/build/building/multi-stage/)

---

PandaStack's free tier uses scale-to-zero on spot nodes for unbeatable economics on dev and hobby apps — and a one-click upgrade to a warm, non-preemptible tier removes cold starts for production. Experiment at [dashboard.pandastack.io](https://dashboard.pandastack.io).