🚨 Kubernetes Error Guide: 'admission webhook denied the request' Blocked Apply

Exact Error Message

$ kubectl apply -f deployment.yaml
Error from server: error when creating "deployment.yaml":
admission webhook "validate.kyverno.svc-fail" denied the request:

resource Deployment/prod/web was blocked due to the following policies:

require-requests-limits:
  autogen-validate-resources: 'validation error: CPU and memory resource
  requests and limits are required. rule autogen-validate-resources failed
  at path /spec/template/spec/containers/0/resources/requests/'

What the Error Means

Kubernetes runs admission webhooks after a request passes authentication and authorization but before the object is persisted. A ValidatingWebhookConfiguration can inspect your object and reject it; a MutatingWebhookConfiguration can rewrite it. When a validating webhook returns a deny, the API server forwards its message back to you verbatim, prefixed with admission webhook "<name>" denied the request.

The cause falls into two broad buckets: either your object genuinely violates a policy (missing resource limits, a disallowed registry, no required label), or the webhook backend itself is unhealthy and its failurePolicy: Fail causes the API server to reject the request because it could not reach the webhook. The webhook name in the message tells you which controller is responsible.

Common Causes

• A policy engine rejecting a real violation — Kyverno, OPA Gatekeeper, or a custom webhook enforcing required labels, resource limits, allowed registries, non-root, or naming rules.
• The webhook backend pod is down or unreachable, and the webhook has failurePolicy: Fail, so the API server denies rather than allowing.
• An expired or mismatched webhook TLS certificate (caBundle), so the API server cannot establish a trusted connection.
• A webhook targeting more than intended because its namespaceSelector/objectSelector is too broad.
• cert-manager, Istio, or another operator’s webhook rejecting a malformed custom resource.
• A timeout — the webhook took longer than timeoutSeconds and failurePolicy: Fail turned the timeout into a denial.

How to Reproduce the Error

Install a simple Kyverno policy that requires resource limits, then apply a Deployment without them:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-requests-limits
spec:
  validationFailureAction: Enforce
  rules:
    - name: validate-resources
      match:
        any:
          - resources:
              kinds: ["Deployment"]
      validate:
        message: "CPU and memory requests and limits are required."
        pattern:
          spec:
            template:
              spec:
                containers:
                  - resources:
                      requests:
                        memory: "?*"
                        cpu: "?*"

kubectl apply -f deployment-without-limits.yaml
# Error: admission webhook "validate.kyverno.svc-fail" denied the request

Diagnostic Commands

# List webhook configurations and which one matches your object
kubectl get validatingwebhookconfigurations
kubectl get mutatingwebhookconfigurations

# Inspect the offending webhook: service, failurePolicy, selectors, rules
kubectl get validatingwebhookconfiguration <NAME> -o yaml | \
  grep -A12 -iE 'clientConfig|failurePolicy|namespaceSelector|rules|timeoutSeconds'

# Check the webhook backend service and its pods are healthy
kubectl get svc -n <WEBHOOK_NS> <WEBHOOK_SVC>
kubectl get pods -n <WEBHOOK_NS> -l <WEBHOOK_LABEL> -o wide

# Read the webhook controller logs for the deny reason (Kyverno example)
kubectl logs -n kyverno deploy/kyverno-admission-controller --tail=50

# See active policies (Kyverno / Gatekeeper)
kubectl get clusterpolicies
kubectl get constraints

# Verify the webhook CA bundle is present (cert health)
kubectl get validatingwebhookconfiguration <NAME> \
  -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | head -c 40; echo

Step-by-Step Resolution

1. Read the webhook name and message. The error names both the webhook (validate.kyverno.svc-fail) and the precise rule that failed. This tells you whether it is a policy violation or a backend problem.

2. If it is a real policy violation, fix the object. Add the missing requirement — here, resource requests and limits:

resources:
  requests: { cpu: "100m", memory: "128Mi" }
  limits:   { cpu: "500m", memory: "256Mi" }

Re-apply and the webhook admits the object.

3. If the webhook backend is down, restore it. Check the backing pods and service; a webhook with failurePolicy: Fail blocks all matching requests when its pods are unavailable:

kubectl get pods -n <WEBHOOK_NS> -l <WEBHOOK_LABEL>
kubectl rollout restart deploy/<WEBHOOK_DEPLOY> -n <WEBHOOK_NS>

4. If a certificate expired, renew it. A stale caBundle makes the API server unable to trust the webhook. For cert-manager-backed webhooks, force a certificate renewal; otherwise re-run the webhook’s cert-injection job.

5. As an emergency unblock, scope or relax the webhook — carefully. If a broken webhook is blocking critical changes, you can temporarily narrow its scope or switch failurePolicy to Ignore, but only as a deliberate break-glass action:

kubectl get validatingwebhookconfiguration <NAME> -o yaml > /tmp/webhook-backup.yaml
# edit failurePolicy: Ignore, apply, fix the backend, then restore

Restore the original configuration as soon as the backend is healthy.

Prevention and Best Practices

• Run policy engines in Audit/Warn mode first, then promote to Enforce once existing workloads comply, to avoid surprise denials.
• Make webhook backends highly available (multiple replicas, PodDisruptionBudgets) so a single pod restart does not block the whole cluster.
• Exclude critical system namespaces (kube-system) from broad webhooks via namespaceSelector to avoid locking yourself out.
• Automate webhook TLS rotation (cert-manager) and alert before certificates expire.
• Prefer failurePolicy: Fail for security-critical policies, but ensure the backend is HA; use Ignore only where availability outranks enforcement.
• Test new policies against a representative set of manifests in CI before rollout. See more in Kubernetes & Helm guides.

Related Errors

• failed calling webhook ... context deadline exceeded — the webhook backend timed out.
• x509: certificate signed by unknown authority — the webhook’s CA bundle does not match its serving cert.
• Internal error occurred: failed calling webhook — the API server could not reach the webhook service at all.
• forbidden: User cannot ... — an authorization (RBAC) denial, which happens before admission webhooks.

Frequently Asked Questions

The deny message names a webhook I did not install. Where is it from? It almost certainly came from an operator or add-on (cert-manager, Istio, Kyverno, Gatekeeper, a cloud provider). Run kubectl get validatingwebhookconfigurations and inspect the named one to find which controller owns it.

Why are unrelated applies suddenly failing after a webhook pod restart? If the webhook has failurePolicy: Fail, the API server denies every matching request while the backend is unreachable. Restore the backend pods or temporarily set failurePolicy: Ignore to unblock, then fix the root cause.

How do I find exactly which rule denied my object? Read the message after denied the request: — policy engines like Kyverno print the failing rule and the JSON path. The webhook controller’s logs give even more detail if the message is truncated.

Is an admission denial the same as an RBAC forbidden error? No. RBAC authorization runs before admission webhooks. A forbidden error means you lack permission; an admission webhook denied error means you are permitted but the object violates a policy or the webhook backend failed.