What Are Preemptible / Spot Nodes?

Cloud providers always have unused capacity sitting idle. Rather than waste it, they sell it cheap — with a catch: they can take it back on short notice. These are spot instances (AWS) or preemptible/Spot VMs (Google Cloud). Used well, they cut compute costs dramatically. Used carelessly, they cause outages. This guide explains how they work and when to use them.

The core idea

A cloud datacenter must keep spare capacity for demand spikes and on-demand customers. Most of the time that buffer is idle. Spot/preemptible nodes let the provider sell that idle capacity at a steep discount — commonly on the order of 60–90% off on-demand prices — in exchange for one condition: the provider can reclaim the node with little warning when it needs the capacity back.

That reclamation is called *preemption* (or *interruption*). You typically get a short notice — Google's preemptible/Spot VMs send a ~30-second termination notice; AWS Spot gives a 2-minute interruption notice — and then the node is gone.

Spot vs on-demand vs reserved

The trade is purely price for reliability. Spot gives up the availability guarantee in exchange for the lowest price.

Why interruption is OK for the right workloads

The whole game is matching spot nodes to workloads that don't care if a node vanishes — because the work can be retried or rescheduled elsewhere. Good fits:

• Stateless workloads behind a load balancer: lose one replica, traffic shifts to others.
• Batch/CI jobs: a build that's interrupted can simply re-run.
• Queue workers: an in-flight message returns to the queue and another worker picks it up.
• Scale-to-zero / bursty services: cheap capacity for spiky, tolerant traffic.

Bad fits:

• Stateful singletons with no failover (a sole database primary).
• Long jobs with no checkpointing that can't afford to restart from zero.
• Latency-critical paths where losing a node mid-request hurts users.

Designing for preemption

If you run on spot, build for interruption from day one:

1. 1Be stateless. Keep state in a managed database or object storage, not on the node.
2. 2Handle the termination signal. Catch the SIGTERM/preemption notice and drain gracefully — finish or requeue in-flight work.
3. 3Spread across many nodes. A single replica on spot is fragile; many replicas absorb individual losses.
4. 4Make work idempotent and retryable. Reprocessing after interruption must be safe.
5. 5Mix node types. A common pattern: a small on-demand baseline plus spot for the elastic, tolerant portion.

# Conceptual graceful-shutdown handler
on SIGTERM:
    stop accepting new work
    finish or requeue in-flight work
    exit cleanly within the notice window

The economics, concretely

Suppose on-demand compute costs $1.00/hr and spot runs at $0.20/hr for the same machine. If your interruptible batch fleet runs 1,000 hours/month:

• On-demand: $1,000/mo
• Spot: $200/mo

That's $800/mo saved on work that doesn't care about interruption — provided your system reschedules interrupted work automatically. The savings are real, but they're only safe if your architecture absorbs preemption transparently.

How PandaStack uses spot nodes

PandaStack runs free-tier apps on spot nodes, combined with a gVisor sandbox for isolation and KEDA scale-to-zero. This is a deliberate, sensible pairing:

• Free-tier apps are typically dev/hobby/low-traffic — exactly the interruption-tolerant profile spot suits.
• Scale-to-zero means an idle app uses no capacity at all; when a request arrives it cold-starts back up.
• Spot's deep discount is what makes a genuinely free tier economically viable.

The honest trade-off, stated plainly: free-tier apps cold-start after idling and run on preemptible capacity. That's perfect for side projects and previews, and not what you'd choose for a latency-critical production service — for which you'd use a paid, non-preemptible tier. (We dig into cold starts in our cold-starts deep dive.)

When NOT to use spot

Don't put your production database primary, your single-replica critical API, or anything where a 30-second eviction means user-visible downtime on spot without a failover story. Spot is a tool for tolerant, redundant, or retryable workloads — not a universal discount you can apply blindly.

References

• [Google Cloud Spot VMs](https://cloud.google.com/compute/docs/instances/spot)
• [AWS EC2 Spot Instances](https://aws.amazon.com/ec2/spot/)
• [Kubernetes graceful pod termination](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination)
• [KEDA: Kubernetes Event-driven Autoscaling](https://keda.sh/docs/latest/concepts/)
• [gVisor sandbox](https://gvisor.dev/docs/)

---

PandaStack's free tier uses spot nodes plus scale-to-zero so it can be genuinely free for dev and hobby apps — and a one-click upgrade moves production workloads onto dedicated capacity. Explore it at [dashboard.pandastack.io](https://dashboard.pandastack.io).