🚨 NGINX Error Guide: '504 Gateway Time-out' upstream timed out

Overview

A 504 Gateway Time-out means NGINX successfully connected to the upstream but did not receive a complete response within the configured timeout window, so it gave up and returned 504 to the client. Unlike a 502 (no/invalid response), a 504 is specifically about time: the backend is alive and talking, just too slow.

NGINX records the precise stage that timed out:

2026/06/23 14:18:42 [error] 3102#3102: *20455 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 203.0.113.42, server: api.example.com, request: "POST /reports/generate HTTP/1.1", upstream: "http://127.0.0.1:8000/reports/generate", host: "api.example.com"

The phrase while reading response header (or while connecting, while sending request) tells you which timeout fired. The number 110 is the OS connection-timeout errno. The cause is either a genuinely slow backend or a timeout set too low for the work being done.

Symptoms

• Browser shows “504 Gateway Time-out” after a long hang (often exactly 60s by default).
• error.log records upstream timed out (110: Connection timed out).
• Slow endpoints (reports, exports, large uploads) fail while fast ones work.
• The backend log shows the request completing a few seconds after NGINX already returned 504.

curl -sI -m 90 https://api.example.com/reports/generate

HTTP/1.1 504 Gateway Time-out
Server: nginx/1.24.0

sudo grep "upstream timed out" /var/log/nginx/error.log | tail -5

[error] 3102#3102: *20455 upstream timed out (110: Connection timed out) while reading response header from upstream, upstream: "http://127.0.0.1:8000/reports/generate"

Common Root Causes

1. proxy_read_timeout is lower than the work takes

The default proxy_read_timeout is 60s. A backend that legitimately needs longer (report generation, heavy query) gets cut off.

grep -RnE 'proxy_read_timeout|proxy_send_timeout|proxy_connect_timeout' \
  /etc/nginx/conf.d/ /etc/nginx/sites-enabled/

[error] 3102#3102: *20455 upstream timed out (110: Connection timed out) while reading response header from upstream, upstream: "http://127.0.0.1:8000/reports/generate"

If no proxy_read_timeout is set for the slow location, NGINX kills the request at 60s even though the backend would have finished.

2. The backend itself is genuinely slow or saturated

The app is up but blocked on a slow database query, an external API, or has exhausted its worker pool, so responses queue.

sudo ss -tn state established '( dport = :5432 )' | wc -l
sudo journalctl -u app-gunicorn --no-pager | grep -iE 'slow|timeout' | tail

[error] upstream timed out (110: Connection timed out) while reading response header from upstream, upstream: "http://127.0.0.1:8000/"

A backlog of established DB connections or app-side slow-query logs lining up with the 504 timestamps confirms a saturated backend.

3. FastCGI / PHP-FPM execution limit shorter than NGINX timeout

For PHP, both fastcgi_read_timeout (NGINX side) and max_execution_time / request_terminate_timeout (PHP-FPM side) apply. The lower one wins.

grep -RnE 'fastcgi_read_timeout' /etc/nginx/
grep -E 'request_terminate_timeout|max_execution_time' \
  /etc/php/8.2/fpm/php.ini /etc/php/8.2/fpm/pool.d/www.conf

[error] 3102#3102: *20480 upstream timed out (110: Connection timed out) while reading response header from upstream, upstream: "fastcgi://unix:/run/php/php8.2-fpm.sock:"

If fastcgi_read_timeout is unset (60s default) but the script needs 90s, NGINX returns 504 first.

4. Slow upstream DNS resolution

When proxy_pass uses a hostname with a resolver, a slow or failing DNS server stalls the connect phase, producing a timeout while connecting.

grep -RnE 'resolver|proxy_pass http' /etc/nginx/conf.d/
dig +short backend.internal @127.0.0.53

[error] 3102#3102: *20512 upstream timed out (110: Connection timed out) while connecting to upstream, upstream: "http://backend.internal:8080/"

while connecting (not while reading) points at the connect/resolve stage rather than the backend’s response time.

5. Large request body times out while being sent

For big POST/uploads, proxy_send_timeout governs how long NGINX waits while streaming the body to a slow upstream.

grep -RnE 'proxy_send_timeout|client_max_body_size' /etc/nginx/conf.d/

[error] 3102#3102: *20530 upstream timed out (110: Connection timed out) while sending request to upstream, upstream: "http://127.0.0.1:8000/upload"

while sending request means the upstream stopped reading the body within proxy_send_timeout.

6. Buffering a huge response to disk stalls

If the response is large and proxy_buffering on; writes to slow temp storage, reading the body can exceed the read timeout even after headers arrive.

grep -RnE 'proxy_buffering|proxy_max_temp_file_size|proxy_temp_path' /etc/nginx/conf.d/
df -h /var/cache/nginx

[warn] 3102#3102: *20560 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/3/00/0000000003 while reading upstream

A full or slow temp path makes large responses crawl and eventually time out.

Diagnostic Workflow

Step 1: Read which stage timed out

sudo grep "upstream timed out" /var/log/nginx/error.log | tail -5

while connecting = connect/DNS; while sending request = body upload; while reading response header/while reading upstream = backend response. This narrows the timeout to fix.

Step 2: Measure how long the backend actually takes

time curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' \
  http://127.0.0.1:8000/reports/generate

If this prints 200 78.4s, the endpoint needs ~78s and any NGINX read timeout below that will 504.

Step 3: Compare NGINX and backend timeouts

grep -RnE 'proxy_read_timeout|proxy_send_timeout|fastcgi_read_timeout' /etc/nginx/
grep -E 'request_terminate_timeout|max_execution_time' \
  /etc/php/8.2/fpm/pool.d/www.conf /etc/php/8.2/fpm/php.ini

The lowest of these is your effective limit. Make sure the backend’s own limit is at least as high as the NGINX timeout you set.

Step 4: Check for backend saturation

sudo ss -tnp state established | grep -c 8000
sudo journalctl -u app-gunicorn --no-pager | grep -iE 'slow|timeout|worker' | tail

If workers are all busy or DB connections are pinned, raising the NGINX timeout only delays the 504 — fix capacity instead.

Step 5: Raise the right timeout for the slow location and reload

# In the specific slow location only, e.g.:
#   location /reports/ {
#       proxy_read_timeout 180s;
#       proxy_send_timeout 180s;
#   }
sudo nginx -t && sudo systemctl reload nginx

Scope the increase to the slow endpoint so a stuck request elsewhere cannot hold a worker for minutes.

Example Root Cause Analysis

POST /reports/generate on api.example.com returns 504 almost exactly 60 seconds after the request starts; faster endpoints are fine.

The error log shows the stage:

[error] 3102#3102: *20455 upstream timed out (110: Connection timed out) while reading response header from upstream, upstream: "http://127.0.0.1:8000/reports/generate"

Timing the backend directly:

time curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' http://127.0.0.1:8000/reports/generate

200 73.9s

The report genuinely takes ~74s, but there is no proxy_read_timeout override for /reports/, so NGINX cuts it off at the 60s default. The backend even finishes and logs a 200 — after NGINX has already returned 504.

Fix: raise the read timeout for just that location, validate, and reload:

# location /reports/ { proxy_read_timeout 180s; proxy_send_timeout 180s; }
sudo nginx -t
sudo systemctl reload nginx

The report now completes through NGINX. (Long-term, moving report generation to an async job is the better design.)

Prevention Best Practices

• Move genuinely long operations (reports, exports, video processing) to a background queue and return a job ID immediately, instead of holding an HTTP request open for minutes.
• Set proxy_read_timeout per slow location rather than globally, so one slow endpoint cannot tie up workers handling the rest of the site.
• Keep NGINX and backend timeouts aligned: fastcgi_read_timeout should be >= PHP request_terminate_timeout, so you get a clean backend error instead of an opaque 504.
• Alert on upstream timed out in error.log and track p95/p99 upstream latency; a creeping p99 predicts 504 storms before users see them.
• Use a static resolver with sane valid= caching for hostname upstreams so DNS stalls don’t masquerade as backend slowness.
• For triage of a sudden wave of 504s, the free incident assistant can read the timeout stage and latency and point at the slow tier. More fixes live in the NGINX guides.

Quick Command Reference

# Which stage timed out?
sudo grep "upstream timed out" /var/log/nginx/error.log | tail -5

# How long does the backend really take?
time curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' http://127.0.0.1:8000/slow-endpoint

# Compare NGINX vs backend timeouts
grep -RnE 'proxy_read_timeout|proxy_send_timeout|fastcgi_read_timeout' /etc/nginx/
grep -E 'request_terminate_timeout|max_execution_time' /etc/php/8.2/fpm/pool.d/www.conf

# Check backend saturation
sudo ss -tnp state established | grep -c 8000

# Validate and reload after raising the timeout
sudo nginx -t && sudo systemctl reload nginx

Conclusion

A 504 Gateway Time-out is a latency problem: NGINX reached the upstream but ran out of patience. The usual root causes:

1. proxy_read_timeout (default 60s) is lower than the work actually takes.
2. The backend is genuinely slow or saturated (DB, worker pool, external API).
3. A FastCGI/PHP-FPM execution limit shorter than the NGINX timeout fires first.
4. Slow or failing DNS resolution stalls the connect phase.
5. proxy_send_timeout fires while streaming a large request body upstream.
6. Buffering a huge response to slow temp storage exceeds the read timeout.

Read which stage the log names — connecting, sending request, or reading — then either fix the slow tier or scope a higher timeout to just that location. Don’t raise timeouts globally; that only hides a capacity problem.