🚨 Kubernetes Error Guide: 'ImagePullBackOff' / 'ErrImagePull' Image Pull Failures

Overview

ErrImagePull and ImagePullBackOff are the states Kubernetes reports when the kubelet cannot pull a container image onto a node. The kubelet asks the container runtime to fetch the image, the pull fails (bad name, auth error, network error, rate limit), and the pod stays at 0/1 with no container ever starting. ErrImagePull is the immediate failure; after a few retries the kubelet backs off and the status becomes ImagePullBackOff (10s, 20s, 40s, up to 5 minutes between attempts).

You will see this in the pod status:

NAME                       READY   STATUS             RESTARTS   AGE
web-6d8c9f7b5c-q4n2t       0/1     ImagePullBackOff   0          1m38s

And the underlying reason in the pod’s events:

Failed to pull image "registry.example.com/web:v2.3.1": rpc error: code = NotFound desc = failed to pull and unpack image: failed to resolve reference: not found

It occurs whenever a pod is scheduled and the image is not already cached on the node — at first deploy, on a new node, after an image tag change, or after registry credentials expire. The pull is node-and-registry specific, so an image that pulls on one node can fail on another with different DNS, credentials, or cached layers.

Symptoms

• Pod stays 0/1 with STATUS of ErrImagePull then ImagePullBackOff.
• RESTARTS stays 0 — the container never starts, so there is nothing to restart.
• kubectl describe pod shows a Failed/Failed to pull image event with the registry’s error.
• No container logs exist yet (kubectl logs returns an error).

kubectl get pods -l app=web

NAME                       READY   STATUS             RESTARTS   AGE
web-6d8c9f7b5c-q4n2t       0/1     ImagePullBackOff   0          1m38s

kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -A8 Events

Events:
  Type     Reason     Age                 From     Message
  ----     ------     ----                ----     -------
  Normal   Scheduled  1m38s               default  Successfully assigned prod/web-6d8c9f7b5c-q4n2t to node-2
  Normal   Pulling    1m37s               kubelet  Pulling image "registry.example.com/web:v2.3.1"
  Warning  Failed     1m35s               kubelet  Failed to pull image "registry.example.com/web:v2.3.1": not found
  Warning  Failed     1m35s               kubelet  Error: ErrImagePull
  Normal   BackOff    24s (x4 over 1m34s) kubelet  Back-off pulling image "registry.example.com/web:v2.3.1"
  Warning  Failed     24s (x4 over 1m34s) kubelet  Error: ImagePullBackOff

Common Root Causes

1. Typo in the image name or tag

A misspelled repository or image name resolves to a reference that does not exist, and the registry returns a not-found error.

kubectl get pod web-6d8c9f7b5c-q4n2t -o jsonpath='{.spec.containers[0].image}'
kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'Failed to pull'

registry.example.com/web-api:v2.3.1
Failed to pull image "registry.example.com/web-api:v2.3.1": failed to resolve reference: not found

The repository is actually web (not web-api). Correct the image string in the Deployment and the pull succeeds.

2. The tag does not exist

The repository is right but the tag was never pushed, or a CI job referenced a tag before it was published. The manifest for that tag is missing.

kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'manifest\|not found'

Failed to pull image "registry.example.com/web:v2.3.1": manifest unknown: manifest unknown

manifest unknown means the repo exists but v2.3.1 does not. Verify the tag with crane ls registry.example.com/web or your registry UI and push or re-tag it.

3. Private registry with no imagePullSecret

Pulling a private image without a registry secret returns an authorization-required error because the request is anonymous.

kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'auth\|unauthorized\|denied'
kubectl get pod web-6d8c9f7b5c-q4n2t -o jsonpath='{.spec.imagePullSecrets}'

Failed to pull image "registry.example.com/web:v2.3.1": failed to authorize: 401 Unauthorized
[]

The empty imagePullSecrets confirms no credentials are attached. Create a docker-registry secret and reference it (see Diagnostic Workflow Step 4).

4. Expired or invalid registry credentials

An imagePullSecret exists but its token/password has expired or is wrong, so the registry rejects the authenticated pull.

kubectl get secret regcred -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d
kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'denied\|forbidden'

{"auths":{"registry.example.com":{"username":"ci","auth":"Y2k6b2xkdG9rZW4="}}}
Failed to pull image "registry.example.com/web:v2.3.1": failed to authorize: 403 Forbidden: access denied

The credentials decode to a stale token. Recreate the secret with a current credential and re-deploy.

5. Registry rate limiting (Docker Hub)

Anonymous or free-tier pulls from Docker Hub are rate-limited; once a node hits the limit, pulls fail with a toomanyrequests error until the window resets.

kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'toomanyrequests\|rate limit'

Failed to pull image "nginx:1.27": toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Authenticate the pulls with an imagePullSecret for Docker Hub, or mirror the image into a private registry to remove the dependency on Hub’s limits.

6. Wrong/unreachable registry or air-gapped node

A node that cannot reach the registry — wrong hostname, firewall, or an air-gapped cluster expected to use a local mirror — fails the pull with a connection or TLS error.

kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'dial\|tls\|connection\|no such host'

Failed to pull image "registry.example.com/web:v2.3.1": dial tcp 203.0.113.10:443: connect: connection timed out

The node has no route to the registry (firewall or air-gap). Point the pod at the in-cluster mirror, or fix the network/firewall path to the registry.

7. Node cannot resolve registry DNS

If the node’s DNS cannot resolve the registry hostname, the pull fails before any TCP connection — distinct from a reachable-but-rejecting registry.

kubectl describe pod web-6d8c9f7b5c-q4n2t | grep -i 'no such host\|server misbehaving'
kubectl get events --field-selector reason=Failed -n prod | tail -3

Failed to pull image "registry.internal/web:v2.3.1": dial tcp: lookup registry.internal on 10.96.0.10:53: no such host

no such host means name resolution failed. Check the registry hostname, the node’s resolver, and any /etc/hosts or split-horizon DNS the cluster relies on.

Diagnostic Workflow

Step 1: Confirm the status and the exact image

kubectl get pods -l app=<APP> -o wide
kubectl get pod <POD> -o jsonpath='{.spec.containers[0].image}'

ErrImagePull/ImagePullBackOff with RESTARTS = 0 confirms a pull failure, not a crash. Note the full image string and the node.

Step 2: Read the pull error from events

kubectl describe pod <POD> | grep -A10 Events
kubectl get events --field-selector involvedObject.name=<POD> --sort-by='.lastTimestamp'

The Failed to pull image message classifies the cause: not found/manifest unknown (name/tag), 401/403 (auth), toomanyrequests (rate limit), dial tcp/no such host (network/DNS).

Step 3: Verify the image name and tag exist

crane ls registry.example.com/web 2>/dev/null
crane manifest registry.example.com/web:v2.3.1 >/dev/null && echo OK || echo MISSING

Confirm both the repository and the specific tag are present in the registry before assuming an auth or network problem.

Step 4: Check or create the imagePullSecret

kubectl get pod <POD> -o jsonpath='{.spec.imagePullSecrets}'
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=<USER> --docker-password=<TOKEN> \
  -n <NS>
kubectl patch serviceaccount default -n <NS> \
  -p '{"imagePullSecrets":[{"name":"regcred"}]}'

Attach the secret to the pod (via the Deployment) or to the ServiceAccount so every pod inherits it.

Step 5: Test connectivity and DNS from a node

kubectl get events --field-selector reason=Failed -n <NS> | tail -5
# from a debug pod on the affected node:
kubectl debug node/<NODE> -it --image=busybox -- nslookup registry.example.com
kubectl debug node/<NODE> -it --image=busybox -- wget -qO- https://registry.example.com/v2/

A failed nslookup is DNS; a failed wget/timeout with successful resolution is routing/firewall.

Example Root Cause Analysis

A new service payments-worker is deployed and stays at 0/1.

kubectl get pods -l app=payments-worker

NAME                                READY   STATUS             RESTARTS   AGE
payments-worker-7b9c4d6f8-mn5kp     0/1     ImagePullBackOff   0          2m05s

The events show an auth failure, not a missing image:

kubectl describe pod payments-worker-7b9c4d6f8-mn5kp | grep -i 'Failed to pull'

Failed to pull image "registry.example.com/payments:v1.4.0": failed to authorize: 401 Unauthorized

So we check whether any pull secret is attached:

kubectl get pod payments-worker-7b9c4d6f8-mn5kp -o jsonpath='{.spec.imagePullSecrets}'

[]

The image is private, but the new Deployment was created without referencing the registry secret that other workloads use. The pull is anonymous, so the registry returns 401.

Fix: attach the existing regcred secret to the ServiceAccount and restart the rollout:

kubectl patch serviceaccount default -n prod \
  -p '{"imagePullSecrets":[{"name":"regcred"}]}'
kubectl rollout restart deployment payments-worker

The next pull authenticates, the image lands, and the pod reaches 1/1 Running.

Prevention Best Practices

• Pin images by immutable digest (@sha256:...) or at least an explicit tag, never latest, so the referenced image cannot disappear or change underneath a deploy.
• Attach imagePullSecrets to the namespace’s ServiceAccount, not just individual pods, so every workload inherits valid credentials.
• Mirror critical public images (Docker Hub base images) into your own registry to avoid rate limits and outages on third-party registries.
• Rotate registry credentials with automation and alert before expiry; an expired token turns every new pod into an ImagePullBackOff.
• Validate that new nodes can resolve and reach the registry as part of node bootstrap, especially in air-gapped clusters that depend on a local mirror.
• Verify the tag exists in the registry in CI before the deploy references it. See more in Kubernetes & Helm guides.

Quick Command Reference

# See the pull failure and the exact image
kubectl get pods -l app=<APP> -o wide
kubectl get pod <POD> -o jsonpath='{.spec.containers[0].image}'

# Why did the pull fail?
kubectl describe pod <POD> | grep -A10 Events
kubectl get events --field-selector involvedObject.name=<POD> --sort-by='.lastTimestamp'

# Verify the image/tag exists in the registry
crane ls registry.example.com/<repo>
crane manifest registry.example.com/<repo>:<tag>

# Inspect / create the pull secret
kubectl get pod <POD> -o jsonpath='{.spec.imagePullSecrets}'
kubectl create secret docker-registry regcred \
  --docker-server=<REGISTRY> --docker-username=<USER> --docker-password=<TOKEN> -n <NS>
kubectl patch serviceaccount default -n <NS> \
  -p '{"imagePullSecrets":[{"name":"regcred"}]}'

# Test DNS/connectivity from the affected node
kubectl debug node/<NODE> -it --image=busybox -- nslookup <REGISTRY>

# Restart the workload after a fix
kubectl rollout restart deployment <DEPLOY>

Conclusion

ErrImagePull and ImagePullBackOff mean the kubelet could not fetch the container image onto the node. The usual root causes:

1. A typo in the image repository or name that resolves to nothing.
2. A tag that was never pushed to the registry (manifest unknown).
3. A private image with no imagePullSecret attached, so the pull is anonymous.
4. Expired or invalid registry credentials returning 403/denied.
5. Registry rate limiting (Docker Hub toomanyrequests).
6. A wrong, firewalled, or air-gapped registry the node cannot reach.
7. A node that cannot resolve the registry’s DNS (no such host).

Read the Failed to pull image event first — its wording (not found, 401, toomanyrequests, no such host) points straight at the category before you touch credentials or networking.