Protecting PVCs From Helm Uninstall With resource-policy keep
A helm uninstall can quietly delete the PVC holding your database. The helm.sh/resource-policy keep annotation protects stateful resources — with one tricky side effect.
- #kubernetes-helm
- #ai
- #helm
- #pvc
- #resource-policy
The first time helm uninstall ate a database, I was the one who ran it. We were tearing down a stale release in a shared namespace, the command exited clean, and about ninety seconds later a teammate asked why the staging Postgres was gone. The chart had templated its own PersistentVolumeClaim, Helm dutifully tracked it as part of the release, and uninstalling the release deleted it along with the Deployment and Service. The PV was on a Delete reclaim policy, so the underlying disk went too. There was no malice and no bug — Helm did exactly what it’s designed to do.
The fix is a single annotation, helm.sh/resource-policy: keep, but it changes ownership semantics in a way that trades one risk for another. Understanding that trade is the difference between a chart that’s safe to uninstall and one that leaves a minefield of orphaned resources behind.
What keep actually does
When you put this annotation on a resource in a chart’s templates, Helm skips that resource during helm uninstall (and during the delete phase of an upgrade that would otherwise remove it):
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-data
annotations:
"helm.sh/resource-policy": keep
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 50Gi
Uninstall the release and the Deployment, Service, and ConfigMap vanish — but the PVC, and the data behind it, survive. That’s exactly what you want for stateful resources.
The catch is in the word Helm uses in its own output: the resource is now orphaned. Helm stops tracking it. It’s no longer part of any release. And that has a consequence people discover at the worst possible time.
The re-install collision
Because the kept resource is no longer owned by Helm, the next time you helm install a release with the same name into the same namespace, Helm tries to create that PVC fresh — and the apiserver rejects it because the object already exists:
Error: INSTALLATION FAILED: rendered manifests contain a resource that
already exists. Unable to continue with install: PersistentVolumeClaim
"postgres-data" in namespace "data" exists and cannot be imported into
the current release: invalid ownership metadata
So keep protected your data, but now your re-install fails until you either delete the orphan (defeating the point) or adopt it back into the release. Adoption means stamping the resource with the labels and annotations Helm uses to claim ownership:
kubectl label pvc postgres-data app.kubernetes.io/managed-by=Helm --overwrite
kubectl annotate pvc postgres-data \
meta.helm.sh/release-name=postgres \
meta.helm.sh/release-namespace=data --overwrite
After that, helm install (or helm upgrade --install) adopts the existing PVC instead of trying to recreate it. This is the part of keep nobody warns you about, and it’s why a blanket “annotate everything” policy backfires.
The CRD trap is worse
Custom resources have an even sharper edge. Helm has two ways to ship CRDs, and they behave completely differently on uninstall:
- CRDs in a
crds/directory are installed once and never deleted by Helm, even without any annotation. - CRDs templated under
templates/are treated as normal resources and are deleted on uninstall — and deleting a CRD cascades to every custom resource of that kind, across all namespaces.
That second case is how a routine uninstall of a monitoring chart can wipe out every ServiceMonitor in the cluster. If a chart templates its CRDs, resource-policy: keep is doing load-bearing safety work, and you need to know it’s there. Check which pattern a chart uses before you trust an uninstall:
Prompt: Here is a Helm chart’s rendered output and its directory layout. Tell me which CRDs are shipped under
crds/versus templated undertemplates/, which resources hold state, and for each stateful resource whetherhelm.sh/resource-policy: keepis present. Flag anything that ahelm uninstallwould delete and take data or custom resources with it. Output a table only — no commands that delete anything.
Output (excerpt): Two CRDs (
servicemonitors,podmonitors) are templated undertemplates/, so uninstall WILL delete them and cascade to all CR instances cluster-wide. Thepostgres-dataPVC has nokeepannotation and would be deleted. Recommend: move CRDs tocrds/or addkeep; addkeepto the PVC with a verified VolumeSnapshot backup.
That kind of mechanical audit across a chart’s manifests is exactly what an AI assistant is good at — it reads the directory structure and rendered YAML, classifies every resource, and never touches the cluster. I keep it firmly in that lane: it annotates manifests and writes the cleanup runbook, and I run every helm and kubectl command myself after reading the output. The model gets chart text, not a kubeconfig.
A retention policy that holds up
The pattern that’s served me well has three parts. First, keep goes only on genuinely stateful resources — PVCs, Secrets holding generated credentials, CRDs you can’t afford to drop — never as a blanket. Second, keep is never the only safety net for data: a kept PVC can still be deleted by a stray kubectl delete, so it’s paired with a real backup, like a CSI VolumeSnapshot on a schedule. Third, there’s a written cleanup step so orphans don’t accumulate:
# Find everything the old release left behind, by instance label
kubectl get pvc,secret,crd \
-l app.kubernetes.io/instance=postgres -A
# Delete a specific orphan deliberately (DESTRUCTIVE — confirm backup first)
kubectl delete pvc postgres-data -n data
Orphaned Secrets matter too. A chart that generates a database password and you keep that Secret leaves live credentials sitting in the namespace after uninstall. Either rotate them or delete them on purpose — don’t let them linger because Helm stopped looking at them.
Wrapping up
helm.sh/resource-policy: keep is the right tool for not deleting your database when you delete a release, but it’s not free: it orphans the resource, which means future installs collide until you adopt it back, and it interacts with the crds/-versus-templates/ distinction in ways that can either protect or destroy every custom resource of a kind. Treat it as a deliberate, per-resource decision backed by a real backup and a cleanup runbook, not a blanket annotation you sprinkle and forget. Let an AI assistant do the audit across the manifests while you keep your hands on every command that could delete data. More retention and storage patterns live in the Kubernetes & Helm guides, and reusable starting points are in the prompt library.
Download the Free 500-Prompt DevOps AI Toolkit
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.