Spot VMs on Google Cloud: when the discount is worth it

Google Cloud sells the same virtual machines at two very different prices. On-demand instances run until you stop them. Spot instances, which replaced the older preemptible offering, run until Google needs the capacity back, at which point they are reclaimed with a very short notice period.

The discount is large enough that it is worth thinking about carefully rather than dismissing. The mistake people make is treating it as a pricing decision. It is an architecture decision that happens to affect pricing.

What actually happens on preemption

When capacity is reclaimed, the instance receives a shutdown signal and then, after a brief grace period measured in seconds rather than minutes, is stopped. That window is enough to flush a buffer, checkpoint progress, deregister from a load balancer or acknowledge that a job needs to be requeued. It is not enough to finish a long computation or to drain a slow connection gracefully.

The older preemptible instances also had a fixed maximum lifetime. Spot instances removed that cap, so a Spot VM can in principle run for a long time, but nothing guarantees it will. Preemption rates vary by machine type, by region and by time, and there is no commitment about frequency.

Two design consequences follow.

Handle the shutdown signal. A shutdown script that checkpoints state or requeues work turns an interruption from data loss into a pause.

Never keep the only copy of anything on a Spot instance. Local state must be reconstructible or replicated somewhere durable.

Workloads that fit naturally

Some workloads are almost designed for this model.

Batch processing. Rendering, transcoding, data transformation, scientific computation. Work is divided into units, units are tracked, an interrupted unit is retried. This is the canonical case and the savings are substantial.

CI and build farms. Builds are short, idempotent and cheap to retry. A build that dies partway through is an inconvenience, not an incident.

Stateless workers pulling from a queue. As long as the queue redelivers unacknowledged messages, worker loss is a normal event the system already handles.

Machine learning training with checkpointing. Training runs that write checkpoints periodically can resume from the last checkpoint. Without checkpointing, a long training run on Spot is a bad idea.

Development and test environments where an occasional restart is tolerable.

Workloads that do not fit

Equally, some things should simply not run on Spot.

Databases and any stateful primary. Anything holding the only copy of data. Long-running jobs without checkpoints, where an interruption at ninety percent means starting again. Latency-sensitive user-facing services without enough on-demand capacity behind them. Anything where the operational cost of an unexpected restart exceeds the money saved.

That last one is the test people skip. If an interruption at three in the morning pages someone, the discount is not a saving.

The mixed pattern, which is usually the right answer

The most durable production pattern is not all-Spot or all-on-demand. It is a baseline of on-demand capacity sized to handle the load you must always serve, with Spot capacity layered on top to absorb everything above that line.

Managed instance groups support this directly. The baseline group runs on-demand and never disappears. The Spot group scales with demand and is allowed to vanish. If Spot capacity is reclaimed en masse, the service degrades to baseline capacity rather than failing.

Sizing the baseline is the real decision. Too small and a mass preemption becomes an outage. Too large and you have paid for the certainty you were trying to avoid buying. A reasonable starting point is the traffic level below which you would consider the service to be genuinely broken, then adjust once you have observed real preemption behaviour.

Reducing the pain of interruptions

A few practices make Spot far more pleasant in production.

• Spread across zones and machine types. Preemption pressure is specific to capacity pools. A workload pinned to one machine type in one zone is exposed to that pool’s pressure; one spread across several is not.
• Make work units small. A job that takes two minutes loses at most two minutes. A job that takes six hours can lose six hours.
• Make retries automatic and idempotent. If a human has to notice and restart something, the model does not work.
• Instrument preemption. Count interruptions. If the rate becomes disruptive, the answer might be a different machine type rather than abandoning Spot.
• Never mix Spot and on-demand in the same instance group if you need to reason about capacity separately. Separate groups make the baseline explicit.

Comparing against committed use

Spot is not the only discount available, and it solves a different problem. Committed use discounts reduce the price of capacity you promise to use for one or three years. They suit steady, predictable load and require no architectural change at all.

The two combine well. Commit to your genuine baseline, because that load exists regardless. Use Spot for the variable portion above it, because that is where interruption is acceptable. On-demand then fills the gap for anything that is neither predictable enough to commit nor tolerant enough for Spot.

Deciding which category each workload belongs to is a more useful exercise than hunting for a single cheapest option, because the answer is usually all three at once.

For the vendor’s own reference on the services involved here, see the Google Cloud documentation.

Questions people ask