🚨 NGINX Error Guide: 'host not found in upstream' — Fix the DNS at Startup

Overview

NGINX refuses to start (or fails nginx -t) and prints this when it cannot resolve a hostname used in a proxy_pass, upstream server, or similar directive at configuration-load time:

nginx: [emerg] host not found in upstream "api-backend.internal" in /etc/nginx/conf.d/proxy.conf:12
nginx: configuration file /etc/nginx/nginx.conf test failed

This is an [emerg] (emergency) error: it is fatal at load time, so a running NGINX will keep serving with its old config but a fresh start or a reload after this appears will fail. The critical fact is when it happens — open-source NGINX resolves hostnames in the config once, at startup, using the system resolver. If DNS can’t answer for that name at that moment, NGINX will not boot. It is not a runtime error; it is a config-validation error caused by DNS.

Symptoms

• sudo systemctl start nginx fails and the unit enters a failed state.
• sudo nginx -t reports [emerg] host not found in upstream "<name>".
• The failure references a specific line in a specific config file.
• A previously working NGINX starts failing after a DNS change, a backend rename, or a reboot where NGINX started before DNS/networking was ready.
• The same hostname resolves fine from the shell with dig/getent, yet NGINX still refuses — a sign of a startup ordering or resolver-scope issue.

Common Root Causes

• Typo or wrong hostname in proxy_pass/upstream — the name genuinely does not exist in DNS.
• DNS not reachable at startup — the name is internal (private zone, Consul, kube-dns) and the resolver couldn’t answer when NGINX loaded.
• Boot ordering — on reboot, NGINX started before the network/DNS or a VPN tunnel was up, so resolution failed for names that work seconds later.
• Backend not yet registered — in service-discovery or container setups the target hasn’t published its DNS record yet.
• Trying to resolve a dynamic name statically — the name is meant to be resolved at runtime, but there is no resolver directive, so NGINX tries (and fails) to resolve it at load time.
• IPv6-only or missing A record — the name has no usable address family NGINX can use.
• /etc/hosts or /etc/resolv.conf misconfiguration on the NGINX host.

Diagnostic Workflow

First confirm exactly which name and line NGINX is choking on:

sudo nginx -t

Then check whether the host itself can resolve that name, using the same path NGINX uses (the system resolver, honoring /etc/hosts and /etc/resolv.conf):

getent hosts api-backend.internal      # uses nsswitch like NGINX does
dig +short api-backend.internal
cat /etc/resolv.conf

If getent fails too, the problem is host-level DNS, not NGINX. If getent succeeds but NGINX still fails at boot, suspect startup ordering. Check whether NGINX started before the network was ready:

systemctl list-dependencies nginx | grep -i network
journalctl -u nginx -b | grep 'host not found'

For an internal name that changes IP and should be resolved at runtime rather than at load, the correct config uses a resolver plus a variable in proxy_pass, which defers resolution to request time:

server {
    resolver 10.0.0.2 valid=30s ipv6=off;   # your internal DNS
    resolver_timeout 5s;

    location /api/ {
        set $api_upstream api-backend.internal;   # variable defers resolution
        proxy_pass http://$api_upstream:8080;
        proxy_set_header Host api-backend.internal;
    }
}

With a variable in proxy_pass, NGINX no longer resolves the name at config load, so nginx -t passes even when the backend isn’t registered yet — resolution happens per request against the resolver.

Example Root Cause Analysis

After a routine reboot, a proxy host failed to bring NGINX back up. systemctl status nginx showed a failed start and nginx -t reported host not found in upstream "vault.service.consul". Yet running getent hosts vault.service.consul from the shell resolved instantly.

The clue was the discrepancy: the name resolved now but not at boot. Consul’s DNS interface (consul agent listening on 127.0.0.1:8600, wired in via systemd-resolved) had not finished starting when NGINX’s unit fired. NGINX tried to statically resolve the Consul name at load time, got nothing, and aborted — seconds before Consul was ready.

Two fixes applied together. First, ordering: add After=consul.service and Wants=network-online.target to a systemd drop-in for NGINX so it starts after service discovery is up. Second, and more robust, switch the vhost to the resolver + variable-in-proxy_pass pattern above, so NGINX defers resolution to request time and never fails to boot merely because a discovery record isn’t published yet. After both, reboots came up clean and a briefly-missing backend produced a runtime 502 (recoverable) instead of a fatal startup failure.

Prevention Best Practices

• For dynamic/internal names, always use a resolver plus a variable in proxy_pass so resolution happens at runtime, not at config load — this keeps nginx -t and startup independent of backend availability.
• Order the NGINX unit After=network-online.target (and after your service-discovery unit) so boot-time DNS is ready before NGINX loads.
• Validate config in CI with nginx -t against a resolver that can actually answer the names, catching typos before deploy.
• Pin truly static backends in /etc/hosts only when their IPs never change; prefer real DNS otherwise.
• Alert on NGINX reload/start failures so a [emerg] doesn’t silently leave stale config running.

Quick Command Reference

# Identify the offending name and line
sudo nginx -t

# Resolve the name the way NGINX does
getent hosts <name>
dig +short <name>
cat /etc/resolv.conf

# Inspect boot ordering and past failures
journalctl -u nginx -b | grep 'host not found'
systemctl list-dependencies nginx | grep -i network

# Apply config after fixing
sudo nginx -t && sudo systemctl reload nginx

Conclusion

host not found in upstream is a startup-time DNS failure, not a traffic problem: NGINX resolves config hostnames once when it loads and refuses to boot if any can’t be answered. Fix the immediate cause (typo, unreachable resolver, or boot ordering), but for any name whose availability or IP can change, adopt the resolver + variable-in-proxy_pass pattern. That decouples NGINX startup from backend readiness, turning a fatal [emerg] into a recoverable runtime condition.