🚨 Azure Error: AKS 'FailedScheduling: 0/3 nodes are available: insufficient cpu' — Cause, Fix, and Troubleshooting Guide

Overview

On AKS, a pod stays Pending and the scheduler reports FailedScheduling when no node can satisfy the pod’s resource requests. The cluster is not broken — there is simply nowhere to place the pod given its CPU/memory requests, node capacity, and any affinity or taint constraints. The literal event from kubectl describe pod:

Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  32s   default-scheduler  0/3 nodes are available: 3 Insufficient cpu.
           preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod.

Variants include Insufficient memory, node(s) had untolerated taint, and node(s) didn't match Pod's node affinity/selector — the count 0/3 tells you none of the three nodes qualified.

Symptoms

• Pods stuck in Pending; kubectl get pods shows them never scheduling.
• kubectl describe pod shows FailedScheduling with Insufficient cpu or Insufficient memory.
• A deployment rollout stalls with some replicas Running and others Pending.
• The cluster autoscaler logs “pod didn’t trigger scale-up” or does nothing because a constraint (not capacity) is the blocker.

Common Root Causes

• Requests too high — the pod’s resources.requests exceed what any single node has free (or exceed a node’s total allocatable).
• Node pool too small — total allocatable CPU/memory across nodes is exhausted by existing workloads plus system pods.
• No autoscaler / at max — the cluster autoscaler is disabled, or already at its --max-count.
• Taints/affinity — nodes are tainted (or the pod’s nodeSelector/affinity excludes them), so “available” nodes are ineligible regardless of capacity.
• Reserved overhead — kubelet system-reserved/kube-reserved and DaemonSets consume allocatable that naive math overlooks.

How to diagnose

Look at the pod’s requests and the scheduling event:

kubectl describe pod <pending-pod> | sed -n '/Events:/,$p'
kubectl get pod <pending-pod> -o jsonpath='{.spec.containers[*].resources}'; echo

Check what each node actually has allocatable and how much is already requested:

kubectl describe nodes | grep -A6 "Allocated resources"
kubectl top nodes 2>/dev/null   # live usage (needs metrics-server)

See node pool sizing and whether the autoscaler is on:

az aks nodepool show \
  --resource-group aks-rg --cluster-name prod-aks --name nodepool1 \
  --query "{count:count, min:minCount, max:maxCount, autoscale:enableAutoScaling, vmSize:vmSize}" -o json

If the reason is a taint or affinity rather than capacity, describe pod says so (untolerated taint / didn't match node affinity) — scaling will not help those.

Fixes

Right-size the requests if they are inflated — lower requests so the pod fits, keeping limits sane:

resources:
  requests:
    cpu: "250m"
    memory: "256Mi"
  limits:
    cpu: "1"
    memory: "512Mi"

Scale the node pool to add capacity:

az aks nodepool scale \
  --resource-group aks-rg --cluster-name prod-aks --name nodepool1 --node-count 5

Enable (or raise) the cluster autoscaler so capacity follows demand:

az aks nodepool update \
  --resource-group aks-rg --cluster-name prod-aks --name nodepool1 \
  --enable-cluster-autoscaler --min-count 3 --max-count 10

Use a larger VM size (or a dedicated node pool) when a single pod’s requests exceed any node’s allocatable — no amount of node count helps if one pod cannot fit on one node:

az aks nodepool add \
  --resource-group aks-rg --cluster-name prod-aks --name bigpool \
  --node-vm-size Standard_D8s_v5 --node-count 2

For taint/affinity blocks, add a matching toleration/nodeSelector to the pod or remove the offending taint — capacity is not the issue there.

What to watch out for

• Requests, not usage, drive scheduling. A pod requesting 4 CPU schedules against requests even if it only uses 200m; over-requesting wastes capacity and causes false Insufficient cpu.
• Allocatable < capacity. System-reserved, kube-reserved, and eviction thresholds shrink what pods can use; do not size against the raw VM specs.
• Autoscaler respects constraints. If the blocker is a taint or affinity, the autoscaler will not add nodes — it only scales for genuine capacity shortfalls.
• DaemonSets take a slice on every node. Log/monitoring agents reduce per-node allocatable; account for them when planning density.

Related

• AKS node NotReady — nodes dropping out reduce schedulable capacity and cause Pending pods.
• AKS ImagePullBackOff — a different pod-stuck state, at pull time rather than scheduling time.
• AllocationFailed — the underlying VMSS cannot get capacity when the node pool tries to scale out.