🚨 Kubernetes Error Guide: 'forbidden: exceeded quota' — Fix ResourceQuota Rejections

Overview

forbidden: exceeded quota is an admission rejection: a namespace has a ResourceQuota object, and the resource you are trying to create would push the namespace over one of its hard caps — total CPU, total memory, number of pods, PVCs, Services, or a custom counted object. The API server refuses the create/update at admission time, so the object is never persisted.

You will see it when applying a manifest or during a rollout:

Error from server (Forbidden): error when creating "deploy.yaml":
pods "web-7d9c" is forbidden: exceeded quota: team-quota,
requested: requests.memory=512Mi,limits.memory=1Gi,
used: requests.memory=7680Mi,limits.memory=15Gi,
limited: requests.memory=8Gi,limits.memory=16Gi

The message is precise: it names the quota (team-quota), what you requested, what is already used, and the limited hard cap. used + requested > limited is the whole story. A subtle variant appears when a quota tracks requests.cpu/requests.memory and a pod omits requests entirely — the quota then rejects the pod for “must specify” the resource.

Symptoms

• kubectl apply returns Forbidden ... exceeded quota and the object is not created.
• A Deployment’s replica count is stuck below desired; new pods never appear.
• kubectl describe replicaset or Deployment events show FailedCreate ... exceeded quota.
• A quota that counts objects (pods, services, persistentvolumeclaims, count/deployments.apps) blocks creation even when CPU/memory are fine.

kubectl get deploy web

NAME   READY   UP-TO-DATE   AVAILABLE   AGE
web    6/10    6            6           22m

kubectl describe rs web-7d9c | grep -A3 Events

Warning  FailedCreate  1m (x9)  replicaset-controller
  Error creating: pods "web-7d9c-xxxx" is forbidden: exceeded quota: team-quota, ...

Common Root Causes

1. Namespace is genuinely at its CPU/memory cap

The sum of pod requests (or limits) across the namespace has reached the quota’s hard value, and the new pod would exceed it. The Deployment controller keeps retrying and never reaches desired replicas.

kubectl describe resourcequota team-quota -n team

Name:            team-quota
Resource         Used    Hard
--------         ----    ----
requests.cpu     3800m   4
requests.memory  7680Mi  8Gi
limits.memory    15Gi    16Gi
pods             18      20

requests.memory is at 7680Mi/8Gi — a 512Mi pod pushes it over. Either raise the quota, lower per-pod requests, or free capacity by scaling something down.

2. Pod omits requests/limits that the quota requires

If a ResourceQuota tracks requests.cpu or limits.memory, every pod in the namespace must declare that resource, or admission rejects it — even if there is plenty of headroom.

Error from server (Forbidden): pods "batch-xyz" is forbidden:
failed quota: team-quota: must specify limits.memory

The fix is to add the missing resources field (often via a LimitRange that supplies defaults).

3. An object-count quota is hit

Quotas can cap object counts, not just compute: pods, services, services.loadbalancers, persistentvolumeclaims, secrets, configmaps, or count/<resource>.<group>. Creating one more of a capped type fails regardless of CPU/memory.

kubectl describe resourcequota team-quota -n team | grep -E 'services|persistentvolumeclaims|pods'

persistentvolumeclaims   5   5
services.loadbalancers   2   2

services.loadbalancers 2/2 blocks a new LoadBalancer Service. Delete an unused one or raise the cap.

4. Scope selector makes the quota apply narrowly (or broadly)

A quota with scopes or scopeSelector (e.g. PriorityClass, BestEffort, NotTerminating) only counts matching pods. A pod unexpectedly hits — or dodges — the quota depending on its priority class or QoS, which surprises people.

kubectl get resourcequota -n team -o jsonpath='{range .items[*]}{.metadata.name}{": "}{.spec.scopeSelector}{"\n"}{end}'

5. Rollout doubles usage during a surge

A rolling update with maxSurge briefly runs extra pods. If the namespace is near its pod or memory cap, the surge pods are rejected and the rollout stalls at partial availability.

kubectl get deploy web -o jsonpath='{.spec.strategy.rollingUpdate}{"\n"}'

{"maxSurge":"25%","maxUnavailable":"25%"}

Diagnostic Workflow

Step 1: Read the rejection message — it contains the answer

kubectl apply -f deploy.yaml

The requested, used, and limited values tell you exactly which resource is over and by how much. For controller-created pods, find it in events instead:

kubectl describe rs <RS> | grep -A3 Events
kubectl get events -n <NS> --field-selector reason=FailedCreate --sort-by='.lastTimestamp'

Step 2: Inspect the quota’s used vs. hard

kubectl get resourcequota -n <NS>
kubectl describe resourcequota <QUOTA> -n <NS>

Every line with Used == Hard is a wall you are hitting.

Step 3: Confirm whether it is compute or object-count

kubectl describe resourcequota <QUOTA> -n <NS> | grep -vE 'requests|limits'

Object caps (pods, services, persistentvolumeclaims) need a different fix than compute caps.

Step 4: Check that pods declare required resources

kubectl get deploy <DEPLOY> -o jsonpath='{.spec.template.spec.containers[*].resources}{"\n"}'
kubectl get limitrange -n <NS>

A missing requests/limits against a quota that requires them is the “must specify” variant; a LimitRange can supply defaults.

Step 5: Account for rollout surge

kubectl get deploy <DEPLOY> -o jsonpath='{.spec.replicas} {.spec.strategy.rollingUpdate}{"\n"}'

If desired replicas plus maxSurge would exceed a cap, lower maxSurge or raise the quota before rolling.

Example Root Cause Analysis

A deploy of api scales to 10 but freezes at 6 replicas.

kubectl describe rs api-6f8 | grep -A2 Events

Warning  FailedCreate  30s (x12)  replicaset-controller
  Error creating: pods "api-6f8-..." is forbidden: exceeded quota: team-quota,
  requested: requests.cpu=500m, used: requests.cpu=3500m, limited: requests.cpu=4

The namespace CPU request budget is 4 cores; 6 pods at 500m use 3000m, plus other workloads bring used to 3500m. Each new pod needs 500m, which would hit 4000m — right at the cap — and the 8th pod tips it over. Confirm:

kubectl describe resourcequota team-quota -n team | grep requests.cpu

requests.cpu     3500m   4

Two clean fixes, depending on intent:

# Option A: the workload really needs the capacity — raise the quota
kubectl patch resourcequota team-quota -n team --type merge \
  -p '{"spec":{"hard":{"requests.cpu":"6"}}}'

# Option B: the pods are over-requesting — lower per-pod CPU requests and redeploy
#   edit resources.requests.cpu from 500m to 300m in the Deployment

After Option A, the ReplicaSet immediately creates the remaining pods and api reaches 10/10. The durable fix is to size quotas from real per-team demand and to right-size pod requests so headroom is not wasted.

Prevention Best Practices

• Pair every ResourceQuota that tracks compute with a LimitRange that supplies default requests/limits, so pods never fail on “must specify.”
• Size quotas from observed usage plus rollout surge headroom, not just steady-state replica counts.
• Alert on used/hard crossing ~80% per namespace so teams raise quotas before a deploy is blocked, not during an incident.
• Lower maxSurge for Deployments in tightly-quota’d namespaces so rolling updates do not transiently exceed caps.
• Right-size pod requests from metrics; over-requesting silently consumes quota that other workloads need.
• Document object-count caps (LoadBalancers, PVCs) alongside compute caps — they cause confusing rejections when compute looks fine. See more in Kubernetes & Helm guides.

Quick Command Reference

# See the rejection (direct apply)
kubectl apply -f <FILE>

# Controller-created pods: find it in events
kubectl describe rs <RS> | grep -A3 Events
kubectl get events -n <NS> --field-selector reason=FailedCreate --sort-by='.lastTimestamp'

# Quota usage vs. hard caps
kubectl get resourcequota -n <NS>
kubectl describe resourcequota <QUOTA> -n <NS>

# Do pods declare required resources?
kubectl get deploy <DEPLOY> -o jsonpath='{.spec.template.spec.containers[*].resources}{"\n"}'
kubectl get limitrange -n <NS>

# Rollout surge vs. caps
kubectl get deploy <DEPLOY> -o jsonpath='{.spec.replicas} {.spec.strategy.rollingUpdate}{"\n"}'

# Fixes
kubectl patch resourcequota <QUOTA> -n <NS> --type merge -p '{"spec":{"hard":{"requests.cpu":"6"}}}'
# or lower per-pod requests / free capacity by scaling something down

Conclusion

forbidden: exceeded quota means a namespace ResourceQuota would be violated by the object you are creating. The rejection message names the quota and prints requested, used, and limited — the arithmetic is the diagnosis. The usual root causes:

1. The namespace is genuinely at its CPU/memory cap.
2. A pod omits requests/limits that the quota requires (“must specify”).
3. An object-count cap (pods, services, PVCs, LoadBalancers) is hit.
4. A scope selector makes the quota apply differently than expected.
5. A rollout’s maxSurge transiently exceeds a cap and stalls.

Read the message, compare used vs. hard, then either raise the quota, right-size requests, or free capacity. For ad-hoc triage, the free incident assistant can turn a quota rejection into a concrete next step.