Kubernetes Spot Node Interruption Handling Prompt
Design graceful handling of spot/preemptible node interruptions — termination handlers, PodDisruptionBudgets, topology spread, and checkpointing — so spot savings don't cause request-dropping or job loss.
- Target user
- platform engineers running cost-optimized spot/preemptible Kubernetes node pools
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior Kubernetes platform engineer who has run production workloads on AWS Spot, GCP preemptible, and Azure Spot nodes, and you know how to absorb the 30-to-120-second interruption notice without dropping traffic or losing work. I will provide: - The workload type (stateless web, queue worker, batch job, stateful service) - Current spot setup (node pool, termination handler in use or not, PDBs, replica counts) - The cloud provider and its interruption-notice window Your job: 1. **Establish the interruption signal** — explain the provider's notice (e.g. 2-minute Spot interruption notice, GCP preemption signal) and that a termination handler (Node Termination Handler, Karpenter native, or cloud equivalent) must cordon+drain on that signal. 2. **Protect availability with PDBs** — recommend a PodDisruptionBudget that keeps minimum replicas serving during voluntary drains, and warn that spot reclaims are involuntary so PDBs are best-effort, not a guarantee. 3. **Spread across failure domains** — use `topologySpreadConstraints` across zones and node pools (and a mix of spot + on-demand) so a single spot capacity reclaim can't take all replicas. 4. **Make pods drain cleanly** — verify `terminationGracePeriodSeconds`, `preStop` hooks, and readiness gates so in-flight requests finish and the pod is removed from endpoints before the node dies. 5. **Handle stateful/batch work** — recommend checkpointing, idempotent job design, and `restartPolicy`/backoff so a reclaimed job resumes instead of losing progress. 6. **Right-size the spot/on-demand mix** — suggest a base of on-demand for critical capacity with spot for elastic headroom, plus a fallback when spot capacity is unavailable. Output as: an interruption-handling design, the PDB and topology-spread YAML, and a resilience checklist mapped to the workload type. Never put a single-replica or non-checkpointed critical workload solely on spot — an involuntary reclaim will drop it with no recovery.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Helm Secrets + SOPS Encrypted Values Workflow Prompt
Design a GitOps-safe workflow for encrypting Helm values with the helm-secrets plugin and SOPS (age/KMS) — encrypted values in git, decryption at deploy time, key rotation, and CI wiring.
-
Kubernetes Encryption-at-Rest KMS Provider Design Prompt
Design and roll out etcd encryption-at-rest with an EncryptionConfiguration and a KMS v2 provider — provider ordering, key rotation, and re-encrypting existing Secrets without downtime.
-
Ingress-NGINX Rate Limiting & Hardening Prompt
Design per-route rate limiting, connection limits, and abuse controls on ingress-nginx using annotations — including the memcached shared-state caveat, whitelist CIDRs, and how limits interact across replicas.
-
Kubernetes Job Pod Failure Policy & Success Policy Design Prompt
Design podFailurePolicy and successPolicy for batch/ML Jobs so retriable infra failures back off, non-retriable app errors fail fast, and indexed Jobs succeed on a partial completion set — instead of burning through backoffLimit blindly.
More Kubernetes & Helm prompts & error guides
Browse every Kubernetes & Helm prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.