☸️ Kubernetes Error Guide: 'Cannot evict pod as it would violate the pod's disruption budget'

Exact Error Message

error when evicting pods/"web-6b9c7d5f4-2xkqz" -n default (will retry after 5s):
Cannot evict pod as it would violate the pod's disruption budget.

During kubectl drain the command loops on this message and never completes, retrying every few seconds indefinitely until you interrupt it.

Overview

A PodDisruptionBudget (PDB) caps how many Pods of a workload may be voluntarily disrupted at once. kubectl drain uses the Eviction API, which honors PDBs. When evicting the next Pod would drop the workload below its minAvailable (or above its maxUnavailable), the API server refuses the eviction and returns Cannot evict pod as it would violate the pod's disruption budget.

This is the PDB doing exactly its job: protecting availability during voluntary disruptions like node drains, cluster upgrades, and autoscaler scale-downs. The drain is blocked, not failed. It usually means the workload cannot currently spare a replica — either because it is already degraded, or because the PDB is stricter than the replica count allows.

Symptoms

• kubectl drain <node> prints Cannot evict pod ... will retry after 5s in a loop and hangs.
• Node stays Ready,SchedulingDisabled (cordoned) but never fully drains.
• Cluster upgrade or autoscaler scale-down stalls on one node.
• kubectl get pdb shows ALLOWED DISRUPTIONS of 0 for the affected workload.

Common Root Causes

1. PDB already at its floor

The workload has exactly minAvailable healthy Pods, so evicting any one would breach the budget. Common when replicas equal minAvailable.

2. Unhealthy or pending replicas

Some Pods are not Ready, so the PDB counts fewer healthy Pods than replicas. The budget treats the workload as already disrupted.

3. Misconfigured PDB

minAvailable: 100% or maxUnavailable: 0 makes any eviction impossible — a permanently blocking budget.

4. Single-replica workload with a PDB

A Deployment with one replica and minAvailable: 1 can never be evicted, because there is no spare to maintain availability during the disruption.

Diagnostic Workflow

Step 1: Find which PDB is blocking

kubectl get pdb --all-namespaces

Look for ALLOWED DISRUPTIONS: 0 — that is the budget refusing evictions.

Step 2: Inspect the blocking budget in detail

kubectl describe pdb web-pdb -n default

Read Min available, Current, Desired, Allowed disruptions, and the selector.

Step 3: Check the health of the selected Pods

kubectl get pods -n default -l app=web -o wide

If Pods are Pending, CrashLoopBackOff, or not Ready, the PDB has no healthy surplus to give.

Step 4: Confirm the replica count versus the budget

kubectl get deploy web -n default -o jsonpath='{.spec.replicas}{"\n"}'

If replicas equals minAvailable, the budget is mathematically un-drainable until you scale up.

Step-by-Step Resolution

1. Identify the PDB and its ALLOWED DISRUPTIONS:

kubectl get pdb -n default

2. Restore replica health first. If Pods are unhealthy, fixing them often lets the drain proceed on its own because the budget regains headroom:

kubectl get pods -n default -l app=web
kubectl rollout status deploy/web -n default

3. If the workload is simply at its floor, scale it up so there is a spare replica to evict:

kubectl scale deploy web -n default --replicas=4

4. Retry the drain, skipping unmanaged Pods and detaching local data where appropriate:

kubectl drain node-1 --ignore-daemonsets --delete-emptydir-data

5. If the PDB itself is misconfigured (maxUnavailable: 0 or minAvailable: 100%), correct it:

kubectl edit pdb web-pdb -n default

6. For a genuine emergency where you must remove the node now and accept the disruption, delete the PDB temporarily and recreate it after maintenance:

kubectl delete pdb web-pdb -n default
# ...perform maintenance, then re-apply the PDB manifest
kubectl apply -f web-pdb.yaml

Prevention

• Always run more replicas than the PDB requires — minAvailable should leave at least one Pod of headroom for voluntary disruptions.
• Prefer maxUnavailable: 1 over minAvailable: N for workloads that scale, so the budget follows the replica count.
• Never set minAvailable: 100% or maxUnavailable: 0 unless you intend to block all drains.
• Give single-replica workloads no PDB, or accept that they will block drains; run at least two replicas if availability matters.
• Keep workloads healthy: a PDB with unhealthy Pods will silently block maintenance windows.
• Use kubectl drain --dry-run=client in change reviews to spot un-drainable nodes before a maintenance window.

Related Errors

• Cannot evict pod ... no running pods to evict — the selector matches nothing, a different eviction issue.
• node(s) had taint {...}, that the pod didn't tolerate — post-drain scheduling, not eviction.
• The node was low on resource: ephemeral-storage — an involuntary eviction by the kubelet, which PDBs do not govern.
• 0/3 nodes are available — capacity for rescheduling drained Pods, a follow-on symptom.

Frequently Asked Questions

Why does kubectl drain retry forever instead of failing? By design. The Eviction API returns a retriable 429-style response so the drain waits for the budget to free up, on the assumption a controller will replace Pods. If the budget can never free up, you must intervene.

How do I see how many disruptions a PDB currently allows? Run kubectl get pdb -n <namespace> and read the ALLOWED DISRUPTIONS column, or kubectl describe pdb for Allowed disruptions.

Can I force the drain past the PDB? kubectl drain has no honest override; the safe path is to scale the workload up or fix unhealthy Pods. Deleting the PDB removes the protection entirely, so only do that for a controlled emergency.

Does the cluster autoscaler respect PDBs? Yes. A node the autoscaler wants to remove will not scale down while a PDB blocks eviction of its Pods, which is a common cause of stalled scale-downs.

For more node-maintenance and availability fixes, see the Kubernetes & Helm guides.