GitLab CI Error Guide: 'could not resolve host' Runner DNS Resolution Failures
Fix GitLab CI 'could not resolve host' and 'dial tcp lookup no such host' DNS errors: runner config.toml dns, dind networking, service aliases, resolv.conf, and extra_hosts.
- #gitlab-cicd
- #troubleshooting
- #errors
- #networking
Overview
When a GitLab CI/CD job fails the moment it tries to reach something over the network — cloning the repo, hitting an internal API, or connecting to a services: database — the cause is frequently DNS, not the remote host itself. The job container asks its resolver for a hostname, the resolver has no answer, and the command aborts before a single byte of TCP is exchanged. Because the failure happens inside the ephemeral job container, the same hostname often resolves fine from your laptop and from the runner host, which makes this one of the more frustrating CI stalls to reproduce.
The error wears several faces depending on which tool hits the wall, but they all mean “this name did not resolve”:
fatal: unable to access 'https://gitlab.example.com/group/app.git/': Could not resolve host: gitlab.example.com
A job talking to a database service by name produces the Go-flavoured variant, where 127.0.0.11 is the embedded Docker DNS resolver inside the job’s network:
dial tcp: lookup postgres on 127.0.0.11:53: no such host
Temporary failure in name resolution
The exit code varies by tool (git, curl, your app), but the signal is constant: the name lookup failed. It is almost always a problem with the runner’s DNS configuration, the job’s container networking, or the hostname you asked for — not with the destination being down.
Symptoms
- A job fails instantly on the first network call (clone,
apt-get,curl, DB connect) rather than timing out after a delay. - The log contains
Could not resolve host,no such host,Temporary failure in name resolution, orbad address. - The same hostname resolves correctly from the runner host shell but not from inside the job container.
- Connections to a
services:host (postgres,mysql,redis) fail while public hostnames may still work — or vice versa. docker:dindjobs can reach the internet but cannot resolve the named service aliases your other jobs use.
# Inside the job: is anything resolving at all?
getent hosts gitlab.example.com || echo "lookup failed"
lookup failed
# What resolver is the job container actually using?
cat /etc/resolv.conf
# /etc/resolv.conf is empty or points at an unreachable server
nameserver 10.0.0.2
An empty resolv.conf, or a nameserver line pointing at an address the runner’s network cannot reach, tells you the job inherited no usable DNS.
Common Root Causes
1. The job container inherits no (or empty) DNS configuration
The Docker executor builds a fresh container per job. If the runner host’s DNS is not propagated into that container — common on hosts using systemd-resolved’s 127.0.0.53 stub, which is meaningless inside a container — /etc/resolv.conf ends up empty or unusable and every lookup fails.
# Inside the job
cat /etc/resolv.conf
nslookup gitlab.example.com
# resolv.conf is empty
;; connection timed out; no servers could be reached
fatal: unable to access 'https://gitlab.example.com/group/app.git/': Could not resolve host: gitlab.example.com
Fix it by giving the Docker executor explicit resolvers in config.toml so every job container gets working DNS:
# /etc/gitlab-runner/config.toml
[[runners]]
[runners.docker]
dns = ["10.0.0.2", "1.1.1.1"]
dns_search = ["internal.example.com"]
2. Connecting to a service by the wrong hostname or alias
services: containers are reachable by their image-derived hostname or an explicit alias:, not by localhost. A job that points its database client at localhost (or at a typo’d alias) gets a lookup failure for the service name, or connects to nothing on the loopback inside its own container.
# .gitlab-ci.yml — the service is reachable as 'postgres', not 'localhost'
test:
services:
- name: postgres:16
alias: postgres
variables:
DATABASE_URL: "postgres://user:pass@postgres:5432/app" # correct
script:
- getent hosts postgres
- psql "$DATABASE_URL" -c '\l'
dial tcp: lookup postgres on 127.0.0.11:53: no such host
If you used localhost, switch to the service alias. If the alias itself is unresolved, confirm the service started (image pulled, no early crash) and that the name matches the alias: exactly.
3. An internal/corporate hostname is not resolvable from the runner’s network
Split-horizon DNS means a name that resolves on the office network or behind a VPN does not resolve from the runner’s subnet. The runner host may reach the public internet fine but have no route to the internal resolver that knows api.internal.example.com.
# Inside the job
curl -sS https://api.internal.example.com/health
getent hosts api.internal.example.com
curl: (6) Could not resolve host: api.internal.example.com
The public DNS resolvers the container uses simply have no record for the internal zone. Point the runner at an internal resolver (dns = [...] in config.toml) that can answer for that zone, or add the internal split DNS server, rather than relying on 1.1.1.1/8.8.8.8 alone.
4. docker:dind has its own network namespace and cannot resolve service aliases
With docker:dind, the Docker daemon runs in a separate container with its own networking. Containers you docker run inside dind do not share the build job’s network, so they cannot resolve postgres/redis service aliases — and without FF_NETWORK_PER_BUILD, services and the job may not share a network at all.
# .gitlab-ci.yml — give every job a shared per-build network
variables:
FF_NETWORK_PER_BUILD: "true"
DOCKER_HOST: "tcp://docker:2376"
DOCKER_TLS_CERTDIR: "/certs"
integration:
services:
- docker:dind
- name: postgres:16
alias: postgres
script:
- getent hosts postgres
ping: bad address 'redis'
dial tcp: lookup postgres on 127.0.0.11:53: no such host
Enable FF_NETWORK_PER_BUILD so the job, its services, and dind share one user-defined network where aliases resolve. Containers started inside dind still need to join that network explicitly (or talk to services via the dind host) to see those names.
5. A transient upstream DNS outage or an overloaded resolver
Sometimes the configuration is correct and the resolver is simply failing right now — an upstream DNS outage, a rate-limited or overloaded internal resolver, or packet loss to the nameserver. These produce intermittent Temporary failure in name resolution that comes and goes between job runs.
# Inside the job — does the resolver answer consistently?
for i in 1 2 3; do getent hosts deb.debian.org || echo "fail $i"; done
dig @10.0.0.2 deb.debian.org +short
fail 1
fail 3
;; connection timed out; no servers could be reached
E: Failed to fetch http://deb.debian.org/... Temporary failure in name resolution
Intermittent failures across otherwise-correct jobs point at the resolver, not your config. Add a second resolver in dns = [...] for redundancy, retry the pipeline, and check the health of the nameserver the runner depends on.
6. A typo’d hostname or a missing /etc/hosts / extra_hosts entry for an IP-only host
If an internal service has no DNS record at all — it is reachable only by IP, or only via a static /etc/hosts mapping — the container has nothing to resolve. The same happens with a simple typo in the hostname. The Docker executor can inject static mappings via extra_hosts.
# Inside the job
getent hosts artifacts.corp.local
# nothing printed — no record exists
curl: (6) Could not resolve host: artifacts.corp.local
Add the static mapping in config.toml (or fix the typo) so the name resolves to the right IP in every job:
# /etc/gitlab-runner/config.toml
[[runners]]
[runners.docker]
extra_hosts = ["artifacts.corp.local:10.20.0.15"]
Diagnostic Workflow
Step 1: Confirm it is DNS, not connectivity
getent hosts gitlab.example.com
nslookup gitlab.example.com
If the name does not resolve, you have a DNS problem — stop chasing firewalls and routes. Could not resolve host and no such host are name-resolution failures that happen before any TCP connection is attempted.
Step 2: Inspect the resolver the job container is using
cat /etc/resolv.conf
dig +short @"$(awk '/nameserver/{print $2; exit}' /etc/resolv.conf)" gitlab.example.com
An empty resolv.conf, a nameserver of 127.0.0.53, or a server that times out tells you the container has no working resolver. That points at the runner’s Docker DNS config (root cause #1).
Step 3: Distinguish public, internal, and service-name failures
getent hosts deb.debian.org # public
getent hosts api.internal.example.com # internal/corporate
getent hosts postgres # services: alias
Public works but internal fails → split-horizon/VPN DNS (#3). Public and internal work but the service alias fails → service networking or dind (#2, #4). Nothing resolves → empty resolver (#1).
Step 4: Check runner config.toml and the per-build network
grep -E 'dns|dns_search|extra_hosts' /etc/gitlab-runner/config.toml
grep -RE 'FF_NETWORK_PER_BUILD' .gitlab-ci.yml
Verify the executor sets dns = [...] for reachable resolvers and that jobs using services: plus docker:dind set FF_NETWORK_PER_BUILD: "true" so aliases land on a shared network.
Step 5: Test resolution against the intended nameserver directly
dig @10.0.0.2 api.internal.example.com +short
for i in 1 2 3; do getent hosts deb.debian.org || echo "transient fail"; done
A direct dig that fails means the chosen resolver cannot answer for that zone (fix the resolver/zone). Intermittent getent failures across retries mean the resolver is flaky or overloaded (root cause #5).
Example Root Cause Analysis
A team migrates their runners to new hosts that use systemd-resolved. Overnight, every pipeline starts failing at the git clone step, even though the GitLab server is healthy and reachable from a browser.
The job log shows the lookup failing before any clone happens:
fatal: unable to access 'https://gitlab.example.com/platform/app.git/': Could not resolve host: gitlab.example.com
From the runner host the name resolves fine, which makes the failure look impossible — until you check what the job container sees:
# Added a debug line at the top of the job
cat /etc/resolv.conf
getent hosts gitlab.example.com || echo "lookup failed"
nameserver 127.0.0.53
lookup failed
The container inherited the host’s systemd-resolved stub resolver at 127.0.0.53. That address only means anything on the host; inside the container it points at a loopback with no resolver listening, so every lookup fails.
Confirm the runner’s Docker executor is not setting explicit DNS:
grep -E 'dns' /etc/gitlab-runner/config.toml
# (no output — no dns configured)
Fix: give the Docker executor real, reachable resolvers and restart the runner so new job containers get a usable resolv.conf:
# /etc/gitlab-runner/config.toml
[[runners]]
[runners.docker]
dns = ["10.0.0.2", "1.1.1.1"]
dns_search = ["internal.example.com"]
sudo gitlab-runner restart
The next pipeline’s job container shows nameserver 10.0.0.2, getent hosts gitlab.example.com resolves, the clone succeeds, and the rest of the pipeline runs.
Prevention Best Practices
- Set explicit
dns = [...](anddns_searchwhere needed) in the Docker executor’sconfig.tomlinstead of relying on the host’ssystemd-resolvedstub, which never works inside containers. - Reference
services:by theiralias:(postgres,redis,mysql) rather thanlocalhost, and document the alias each job expects so a rename does not silently break lookups. - Enable
FF_NETWORK_PER_BUILD: "true"for jobs that combineservices:withdocker:dindso the job, services, and daemon share one network where aliases resolve. - Point runners at an internal/split-horizon resolver that can answer for your private zones, and keep a redundant second
dnsentry so a single nameserver outage does not stall every pipeline. - Use
extra_hostsfor IP-only internal services that have no DNS record, and review hostnames in.gitlab-ci.ymlfor typos during code review. - For fast triage of a DNS-failure storm across pipelines, the free incident assistant can read the resolution error and point at the likely runner or networking cause. More pipeline fixes live in the GitLab CI/CD guides.
Quick Command Reference
# Confirm it is DNS and see what the container resolves
getent hosts gitlab.example.com
nslookup gitlab.example.com
cat /etc/resolv.conf
# Classify the failure: public vs internal vs service alias
getent hosts deb.debian.org
getent hosts api.internal.example.com
getent hosts postgres
# Query the intended resolver directly
dig @10.0.0.2 api.internal.example.com +short
for i in 1 2 3; do getent hosts deb.debian.org || echo "transient fail"; done
# Inspect runner DNS and per-build networking
grep -E 'dns|dns_search|extra_hosts' /etc/gitlab-runner/config.toml
grep -RE 'FF_NETWORK_PER_BUILD' .gitlab-ci.yml
# config.toml: give job containers working DNS and static hosts
# [runners.docker]
# dns = ["10.0.0.2", "1.1.1.1"]
# dns_search = ["internal.example.com"]
# extra_hosts = ["artifacts.corp.local:10.20.0.15"]
# Apply config changes
sudo gitlab-runner restart
Conclusion
A “could not resolve host” or “no such host” job failure is GitLab telling you a name lookup failed inside the job container, before any connection was made. The usual root causes:
- The job container inherited no usable DNS (empty
resolv.confor the host’ssystemd-resolvedstub); fix withdns = [...]inconfig.toml. - The job used the wrong hostname for a service —
localhostinstead of thepostgres/redisalias:. - An internal or corporate hostname is unreachable from the runner’s network due to split-horizon or VPN-only DNS.
docker:dindruns in its own network namespace and cannot resolve service aliases withoutFF_NETWORK_PER_BUILD.- A transient upstream DNS outage or overloaded resolver produces intermittent
Temporary failure in name resolution. - A typo’d hostname or an IP-only internal service with no DNS record, needing a corrected name or an
extra_hostsentry.
Confirm it is DNS first with getent hosts and /etc/resolv.conf, then decide whether the fix is reachable resolvers in config.toml, the correct service alias, a per-build network, or a static extra_hosts mapping — and the stuck pipeline starts resolving again.
Download the Free 500-Prompt DevOps AI Toolkit
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.