Skip to content
DevOps AI ToolKit
Newsletter
All guides
AI for Kubernetes & Helm By James Joyner IV · · 10 min read

Protecting PVCs From Helm Uninstall With resource-policy keep

A helm uninstall can quietly delete the PVC holding your database. The helm.sh/resource-policy keep annotation protects stateful resources — with one tricky side effect.

  • #kubernetes-helm
  • #ai
  • #helm
  • #pvc
  • #resource-policy

The first time helm uninstall ate a database, I was the one who ran it. We were tearing down a stale release in a shared namespace, the command exited clean, and about ninety seconds later a teammate asked why the staging Postgres was gone. The chart had templated its own PersistentVolumeClaim, Helm dutifully tracked it as part of the release, and uninstalling the release deleted it along with the Deployment and Service. The PV was on a Delete reclaim policy, so the underlying disk went too. There was no malice and no bug — Helm did exactly what it’s designed to do.

The fix is a single annotation, helm.sh/resource-policy: keep, but it changes ownership semantics in a way that trades one risk for another. Understanding that trade is the difference between a chart that’s safe to uninstall and one that leaves a minefield of orphaned resources behind.

What keep actually does

When you put this annotation on a resource in a chart’s templates, Helm skips that resource during helm uninstall (and during the delete phase of an upgrade that would otherwise remove it):

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
  annotations:
    "helm.sh/resource-policy": keep
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 50Gi

Uninstall the release and the Deployment, Service, and ConfigMap vanish — but the PVC, and the data behind it, survive. That’s exactly what you want for stateful resources.

The catch is in the word Helm uses in its own output: the resource is now orphaned. Helm stops tracking it. It’s no longer part of any release. And that has a consequence people discover at the worst possible time.

The re-install collision

Because the kept resource is no longer owned by Helm, the next time you helm install a release with the same name into the same namespace, Helm tries to create that PVC fresh — and the apiserver rejects it because the object already exists:

Error: INSTALLATION FAILED: rendered manifests contain a resource that
already exists. Unable to continue with install: PersistentVolumeClaim
"postgres-data" in namespace "data" exists and cannot be imported into
the current release: invalid ownership metadata

So keep protected your data, but now your re-install fails until you either delete the orphan (defeating the point) or adopt it back into the release. Adoption means stamping the resource with the labels and annotations Helm uses to claim ownership:

kubectl label pvc postgres-data app.kubernetes.io/managed-by=Helm --overwrite
kubectl annotate pvc postgres-data \
  meta.helm.sh/release-name=postgres \
  meta.helm.sh/release-namespace=data --overwrite

After that, helm install (or helm upgrade --install) adopts the existing PVC instead of trying to recreate it. This is the part of keep nobody warns you about, and it’s why a blanket “annotate everything” policy backfires.

The CRD trap is worse

Custom resources have an even sharper edge. Helm has two ways to ship CRDs, and they behave completely differently on uninstall:

  • CRDs in a crds/ directory are installed once and never deleted by Helm, even without any annotation.
  • CRDs templated under templates/ are treated as normal resources and are deleted on uninstall — and deleting a CRD cascades to every custom resource of that kind, across all namespaces.

That second case is how a routine uninstall of a monitoring chart can wipe out every ServiceMonitor in the cluster. If a chart templates its CRDs, resource-policy: keep is doing load-bearing safety work, and you need to know it’s there. Check which pattern a chart uses before you trust an uninstall:

Prompt: Here is a Helm chart’s rendered output and its directory layout. Tell me which CRDs are shipped under crds/ versus templated under templates/, which resources hold state, and for each stateful resource whether helm.sh/resource-policy: keep is present. Flag anything that a helm uninstall would delete and take data or custom resources with it. Output a table only — no commands that delete anything.

Output (excerpt): Two CRDs (servicemonitors, podmonitors) are templated under templates/, so uninstall WILL delete them and cascade to all CR instances cluster-wide. The postgres-data PVC has no keep annotation and would be deleted. Recommend: move CRDs to crds/ or add keep; add keep to the PVC with a verified VolumeSnapshot backup.

That kind of mechanical audit across a chart’s manifests is exactly what an AI assistant is good at — it reads the directory structure and rendered YAML, classifies every resource, and never touches the cluster. I keep it firmly in that lane: it annotates manifests and writes the cleanup runbook, and I run every helm and kubectl command myself after reading the output. The model gets chart text, not a kubeconfig.

A retention policy that holds up

The pattern that’s served me well has three parts. First, keep goes only on genuinely stateful resources — PVCs, Secrets holding generated credentials, CRDs you can’t afford to drop — never as a blanket. Second, keep is never the only safety net for data: a kept PVC can still be deleted by a stray kubectl delete, so it’s paired with a real backup, like a CSI VolumeSnapshot on a schedule. Third, there’s a written cleanup step so orphans don’t accumulate:

# Find everything the old release left behind, by instance label
kubectl get pvc,secret,crd \
  -l app.kubernetes.io/instance=postgres -A

# Delete a specific orphan deliberately (DESTRUCTIVE — confirm backup first)
kubectl delete pvc postgres-data -n data

Orphaned Secrets matter too. A chart that generates a database password and you keep that Secret leaves live credentials sitting in the namespace after uninstall. Either rotate them or delete them on purpose — don’t let them linger because Helm stopped looking at them.

Wrapping up

helm.sh/resource-policy: keep is the right tool for not deleting your database when you delete a release, but it’s not free: it orphans the resource, which means future installs collide until you adopt it back, and it interacts with the crds/-versus-templates/ distinction in ways that can either protect or destroy every custom resource of a kind. Treat it as a deliberate, per-resource decision backed by a real backup and a cleanup runbook, not a blanket annotation you sprinkle and forget. Let an AI assistant do the audit across the manifests while you keep your hands on every command that could delete data. More retention and storage patterns live in the Kubernetes & Helm guides, and reusable starting points are in the prompt library.

Free download · 368-page PDF

Download the Free 500-Prompt DevOps AI Toolkit

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.