🚨 Kubernetes Error Guide: '0/5 nodes are available' FailedScheduling Pending Pods

Exact Error Message

When the kube-scheduler cannot place a pod, the pod stays Pending and a FailedScheduling event is recorded with a per-predicate breakdown of why each node was rejected:

Events:
  Type     Reason            Age                From               Message
  ----     ------            ----               ----               -------
  Warning  FailedScheduling  41s (x3 over 2m)   default-scheduler  0/5 nodes are available: 3 Insufficient cpu, 2 node(s) had untolerated taint {dedicated: gpu}. preemption: 0/5 nodes are available: 3 No preemption victims found for incoming pod, 2 Preemption is not helpful for scheduling.

The leading 0/5 nodes are available is the headline: zero of five candidate nodes passed every scheduling check. Everything after the colon is a count-prefixed list of the reasons nodes were filtered out.

What the Error Means

The scheduler runs in two phases for every Pending pod: filtering (which nodes are even feasible) and scoring (which feasible node is best). FailedScheduling is a filtering failure — no node survived the filter plugins, so scoring never ran and the pod has nowhere to go.

The message aggregates the rejection reason from each filter plugin and prefixes it with the number of nodes that hit that reason. 3 Insufficient cpu means three nodes did not have enough allocatable CPU; 2 node(s) had untolerated taint means two nodes carry a taint the pod does not tolerate. The counts add up to the total node count (here 3 + 2 = 5), so you can confirm you have accounted for every node.

This is the general FailedScheduling triage page. Two of the most common single causes have dedicated deep-dives — see the sibling guides on insufficient cpu/memory and untolerated taints. Use this page to read the breakdown string and route to the right fix.

Common Causes

• Insufficient cpu/memory — the sum of the pod’s requests exceeds any node’s allocatable capacity (not total capacity).
• Untolerated taints — nodes carry a taint (NoSchedule/NoExecute) the pod lacks a matching toleration for.
• Node affinity / nodeSelector no match — nodeSelector or requiredDuringScheduling affinity references labels no node has.
• Pod anti-affinity — podAntiAffinity forbids co-locating with pods already running on every node.
• Volume node affinity / topology — a bound PV is pinned to a zone/node, and no feasible node is in that topology.
• Unschedulable / cordoned nodes — nodes marked SchedulingDisabled by kubectl cordon are excluded.
• Too many pods — a node already runs its maxPods (default 110), so it reports Too many pods.

How to Reproduce the Error

Create a deployment that requests more CPU than any node can offer:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cpu-hog
spec:
  replicas: 1
  selector:
    matchLabels: { app: cpu-hog }
  template:
    metadata:
      labels: { app: cpu-hog }
    spec:
      containers:
        - name: app
          image: registry.k8s.io/pause:3.9
          resources:
            requests:
              cpu: "8"

kubectl apply -f cpu-hog.yaml
kubectl get pods -l app=cpu-hog

NAME                       READY   STATUS    RESTARTS   AGE
cpu-hog-5c7d9f8b6d-2xqzt   0/1     Pending   0          18s

The pod never leaves Pending and kubectl describe shows 0/N nodes are available: N Insufficient cpu.

Diagnostic Commands

# Confirm the pod is Pending and has not been assigned a node
kubectl get pod <POD> -o wide

# Read the full FailedScheduling breakdown
kubectl describe pod <POD> | grep -A6 Events

# What does the pod request?
kubectl get pod <POD> -o jsonpath='{.spec.containers[*].resources.requests}'

# What can each node actually offer (allocatable)?
kubectl describe nodes | grep -A6 'Allocatable\|Allocated resources'

# Inspect taints, labels, and unschedulable flag per node
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints,UNSCHED:.spec.unschedulable
kubectl describe node <NODE> | grep -i 'taint\|Non-terminated\|Allocated'

The describe pod Events block is the single most important output — read its count-prefixed clauses left to right.

Step-by-Step Resolution

1. Parse the breakdown string. Split the message on commas. Each clause is <count> <reason>. Confirm the counts sum to the total node count so nothing is hidden.

0/5 nodes are available: 3 Insufficient cpu, 2 node(s) had untolerated taint {dedicated: gpu}.

2. Fix Insufficient cpu/memory. Compare requests to allocatable. Lower the request, right-size the workload, or add/scale nodes:

kubectl get pod <POD> -o jsonpath='{.spec.containers[*].resources.requests}'
# -> {"cpu":"8"}  but largest node Allocatable cpu is 3920m -> impossible

Reduce the request to fit, or add a larger node. See the insufficient cpu/memory guide.

3. Fix untolerated taint. Add a matching toleration to the pod spec, or remove the taint from nodes you want to use:

spec:
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "gpu"
      effect: "NoSchedule"

Details in the untolerated taint guide.

4. Fix affinity / nodeSelector mismatch. didn't match Pod's node affinity/selector means no node has the required label. List labels and either label a node or relax the selector:

kubectl get nodes --show-labels | grep -o 'disktype=[^,]*'
kubectl label node <NODE> disktype=ssd

5. Fix pod anti-affinity / topology spread. node(s) didn't match pod anti-affinity rules means every node already hosts a conflicting pod. Add nodes, or relax requiredDuringScheduling to preferredDuringScheduling.

6. Fix volume node affinity. node(s) had volume node affinity conflict means the pod’s bound PV is in a zone none of the feasible nodes are in. Schedule into that zone or recreate the PVC with WaitForFirstConsumer (see the PVC not bound guide).

7. Fix cordoned / full nodes. Uncordon a node, or address Too many pods:

kubectl uncordon <NODE>

Prevention and Best Practices

• Set realistic requests based on observed usage; oversized requests silently make pods unschedulable as the cluster fills.
• Run a cluster autoscaler so Insufficient cpu/memory triggers a new node instead of an indefinite Pending.
• Keep tolerations in sync with taints: when you taint a node pool, update every workload meant to run there.
• Prefer preferredDuringScheduling over requiredDuringScheduling for affinity unless placement is truly mandatory.
• Alert on pods Pending longer than a few minutes — a stuck FailedScheduling will not self-heal without capacity or spec changes.
• Use WaitForFirstConsumer StorageClasses so volume topology is chosen after scheduling, avoiding volume affinity conflicts. More in Kubernetes & Helm guides.

Related Errors

• Insufficient cpu / memory — deep-dive on requests vs allocatable.
• Untolerated taint — taints and tolerations.
• PersistentVolumeClaim not bound — the volume topology cause.
• ImagePullBackOff — a scheduled pod that then fails to pull its image.

Frequently Asked Questions

Why does the message say 0/5 when I have more than five nodes? The scheduler only counts nodes that are candidates after preliminary filtering (Ready, not tainted out at the node-info level). Control-plane and NotReady nodes are excluded, so 0/5 can appear in a larger cluster.

The breakdown lists several reasons — which do I fix first? You must satisfy all of them; a node has to pass every filter. Fix the reason covering the most nodes first, then re-check the new breakdown, since clearing one reason often reveals the next constraint.

My pod requests 500m CPU and the node shows 2 cores free, but it still says Insufficient cpu. Why? The scheduler counts requests already reserved by other pods, not live usage. A node can be idle yet have all its CPU reserved by requests. Compare against Allocated resources in kubectl describe node, not top.

Does FailedScheduling ever resolve on its own? Only if capacity appears — a cluster autoscaler adds a node, a pod elsewhere terminates and frees requests, or a node is uncordoned. The scheduler re-queues Pending pods and retries automatically when cluster state changes.

How is FailedScheduling different from Pending due to image pull? A FailedScheduling pod has no node assigned (NODE is <none>). A pod stuck pulling an image is already scheduled and shows a node; that is ImagePullBackOff, not a scheduling failure.