🚨 Kubernetes Error Guide: 'Readiness probe failed: statuscode: 503' — Fix Unready Pods

Overview

A pod stays 0/1 READY and its events show the readiness probe returning HTTP 503. Unlike a liveness failure (which restarts the container), a failing readiness probe does not kill the pod — it just keeps the pod out of Service endpoints, so it receives no traffic. A 503 specifically means the app is answering the probe but is deliberately reporting itself as not ready: its readiness endpoint returns 503 because a dependency check failed, warmup is incomplete, or the app is shedding load.

The literal event looks like this:

Warning  Unhealthy  kubelet  Readiness probe failed: HTTP probe failed with statuscode: 503

The container is Running but never Ready, so it is excluded from its Service’s endpoints and the Deployment cannot mark the rollout as available. The key insight: 503 is the app’s own answer. Something inside the readiness handler is choosing to report unready — find what, rather than assuming the probe is misconfigured.

Symptoms

• Pod is Running but READY 0/1, with a climbing count of Unhealthy events.
• Events read Readiness probe failed: HTTP probe failed with statuscode: 503.
• The pod is absent from kubectl get endpoints <svc>.
• A rollout is stuck because new pods never become Ready.

kubectl get pod api-7d9c-abcde

NAME             READY   STATUS    RESTARTS   AGE
api-7d9c-abcde   0/1     Running   0          3m

kubectl describe pod api-7d9c-abcde | grep -A1 Readiness

  Warning  Unhealthy  Readiness probe failed: HTTP probe failed with statuscode: 503

Common Root Causes

1. A dependency check inside the readiness handler is failing

Many apps make /ready return 503 until a downstream dependency (DB, cache, upstream API) is reachable. If that dependency is down, the pod correctly reports unready.

kubectl exec api-7d9c-abcde -- curl -s -o /dev/null -w '%{http_code}' localhost:8080/ready
kubectl logs api-7d9c-abcde | tail -5

503
2026-07-09T10:12:04Z readiness: postgres ping failed: connection refused

Fix the dependency (or make the readiness check reflect only what the pod truly needs to serve).

2. Warmup not finished (cache load, JIT, migrations)

An app that loads a large cache or runs migrations at boot returns 503 from /ready until warmup completes. If initialDelaySeconds/failureThreshold are too tight, it looks like a failure when it just needs more time.

kubectl describe pod api-7d9c-abcde | grep -A4 Readiness

Readiness:  http-get http://:8080/ready delay=2s timeout=1s period=5s #success=1 #failure=3

3. The app is shedding load / at capacity

Some frameworks return 503 when a connection/thread pool is saturated, so a healthy-but-overloaded pod flaps out of readiness. This points at capacity, not a bug.

4. Wrong readiness path hitting a 503 route

If the probe path points at an endpoint that returns 503 (a maintenance route, or a path guarded by an auth layer returning 503), the probe fails even though the app is fine.

kubectl get pod api-7d9c-abcde -o jsonpath='{.spec.containers[0].readinessProbe.httpGet}'

{"path":"/","port":8080}

5. Ingress/proxy in front returning 503, not the app

If the probe goes through a sidecar/proxy that itself returns 503 before the app is registered, you see 503 during the mesh’s own startup window.

Diagnostic Workflow

Step 1: Confirm it is readiness, not liveness

kubectl describe pod <POD> | grep -E 'Readiness|Liveness'

Readiness failures keep the pod out of endpoints; liveness failures restart it. A 503 with no restarts is readiness.

Step 2: Hit the readiness endpoint from inside the pod

kubectl exec <POD> -- curl -si localhost:<port><path>

The body/headers usually explain why the app reports 503 (which dependency, or “warming up”).

Step 3: Read the app logs at probe time

kubectl logs <POD> --tail=30

Look for the dependency name or warmup message tied to the 503.

Step 4: Check the probe timing vs real startup

kubectl get pod <POD> -o jsonpath='{.spec.containers[0].readinessProbe}'

If startup legitimately takes longer than the probe allows, add a startupProbe or raise initialDelaySeconds/failureThreshold.

Step 5: Confirm endpoint membership after the fix

kubectl get endpoints <SVC>
kubectl get pod <POD> -o wide

Example Root Cause Analysis

A new rollout of orders-api stalls; new pods never become Ready:

kubectl get pods -l app=orders-api

NAME                    READY   STATUS    RESTARTS   AGE
orders-api-59c8-2fk9d   0/1     Running   0          4m

kubectl describe pod orders-api-59c8-2fk9d | grep -A1 Readiness

  Warning  Unhealthy  Readiness probe failed: HTTP probe failed with statuscode: 503

Hitting the endpoint from inside the pod shows the reason:

kubectl exec orders-api-59c8-2fk9d -- curl -si localhost:8080/ready

HTTP/1.1 503 Service Unavailable
{"ready":false,"checks":{"redis":"ok","postgres":"connection refused"}}

The readiness handler pings Postgres, which is refusing connections. Checking the DB service:

kubectl get endpoints orders-db

NAME        ENDPOINTS   AGE
orders-db   <none>      4m

The database has no endpoints — its own pods are down after a failed migration. The 503 is correct: the app should not receive traffic without its database. Fixing the DB restores readiness:

kubectl rollout restart statefulset orders-db
kubectl rollout status statefulset orders-db
kubectl get pods -l app=orders-api -w

Once Postgres is back, /ready returns 200, the pods join the Service endpoints, and the orders-api rollout completes.

Prevention Best Practices

• Make readiness checks reflect only what a pod needs to serve requests, and log which dependency failed so a 503 is self-explaining.
• Use a startupProbe for slow-warming apps so readiness is not evaluated until warmup completes, instead of loosening the readiness probe.
• Distinguish liveness from readiness: never make liveness depend on external dependencies, or a dependency outage will restart-loop healthy pods.
• Alert on pods Running but not Ready for longer than a threshold, since they silently drop out of traffic without crashing.
• Load-test so readiness flapping under capacity pressure is a capacity decision, not a surprise. See more in Kubernetes & Helm guides.

Quick Command Reference

# Confirm readiness (not liveness) failure
kubectl describe pod <POD> | grep -E 'Readiness|Liveness'

# Hit the readiness endpoint from inside the pod
kubectl exec <POD> -- curl -si localhost:<port><path>

# App logs around the 503
kubectl logs <POD> --tail=30

# Probe configuration
kubectl get pod <POD> -o jsonpath='{.spec.containers[0].readinessProbe}'

# Endpoint membership after the fix
kubectl get endpoints <SVC>

Conclusion

Readiness probe failed: HTTP probe failed with statuscode: 503 means the app answered the probe but reported itself as not ready. The usual causes:

1. A dependency check inside the readiness handler is failing.
2. Warmup (cache/migrations) has not finished.
3. The app is shedding load at capacity.
4. The probe points at a path that returns 503.
5. A sidecar/proxy in front returns 503 during its own startup.

Curl the readiness endpoint from inside the pod — the response almost always names the reason. For ad-hoc triage, the free incident assistant can turn a readiness 503 and its logs into the specific dependency to fix.