Failed Helm Upgrade Recovery Runbook Prompt
Recover from a failed or partially-applied Helm upgrade by reading release history and status, deciding between rollback, --force, and manual repair, without losing data or compounding the failure.
- Target user
- SREs and release engineers
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a senior release engineer recovering a Helm release that failed mid-upgrade. Diagnose the release state before touching anything, then choose the safest path back to healthy. I will provide: - `helm history <release>` and `helm status <release>` output (revision, status: failed/pending-upgrade/deployed) - The error from the failed `helm upgrade` (timeout, immutable field, hook failed, resource conflict, "another operation in progress") - Relevant `kubectl get/describe` for the workloads the upgrade touched - Whether the chart manages stateful resources (PVCs, StatefulSets, CRDs) Your job: 1. **Classify the failure** — distinguish a stuck `pending-upgrade`/`pending-install` lock from a clean `failed` revision from a hook failure; explain what each implies. 2. **Diagnose the trigger** — immutable field change (e.g. selector/volumeClaimTemplates), failed pre/post hook, atomic timeout, or an out-of-band kubectl edit causing drift. 3. **Choose the recovery** — rollback to the last good revision (`helm rollback`), re-run with corrected values, `--force` (and its risks: replace can disrupt), or manual resource repair; justify the choice and what it does to live traffic. 4. **Clear stuck locks safely** — explain how a release stuck in pending state blocks future operations and the careful way to resolve it without corrupting release secrets. 5. **Protect state** — flag any step that could delete/recreate PVCs, StatefulSets, or CRDs and how to avoid data loss; note resource-policy keep annotations. 6. **Prevent recurrence** — recommend `--atomic`, `--timeout`, hook hardening, and a values diff/preview before the next upgrade. Output: a numbered recovery runbook (assess, decide, execute, verify), the exact commands, the blast radius of each, and the rollback-of-the-rollback if it fails.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Helm Release Rollback & Stuck Release Debug Prompt
Recover from a Helm release stuck in `pending-install` / `pending-upgrade` / `failed`, roll back safely, and avoid Helm-secret bloat that breaks future operations.
-
Helm Stuck Pending Release State Recovery Prompt
Recover a Helm release stuck in pending-install, pending-upgrade, or pending-rollback after a failed or interrupted operation — without corrupting release history or losing live workloads.
-
Helm Chart.lock Dependency Drift Reconciliation Prompt
Resolve Chart.yaml vs Chart.lock dependency drift and 'found in Chart.yaml, but missing in charts/ directory' failures by reasoning about version ranges, the lock digest, and helm dependency build vs update.
-
Helm Release History Pruning and max-history Prompt
Stop Helm release history from bloating etcd with hundreds of Secret/ConfigMap revisions — set --history-max, prune safely, and avoid hitting the 1MB Secret size limit or slow upgrades.
More Kubernetes & Helm prompts & error guides
Browse every Kubernetes & Helm prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.