DNS Troubleshooting With Kali Linux

More outages get blamed on “the application” than any single component deserves, and a large share of those are really name-resolution failures wearing an application costume. A service that “can’t reach the database,” a deploy that “can’t pull the image,” an API client throwing connection errors — dig the name it’s using and you’ll frequently find DNS handing back the wrong address, no address, or a stale one. This lesson makes you fluent in the two tools that settle those questions on a modern Linux box: dig for asking DNS precise questions, and resolvectl for seeing which resolver your system is actually using. We’ll define every record type you’ll meet, read real output line by line, and walk the specific failure signatures — including the container ones that trip up Docker and Kubernetes users.

Everything here builds directly on Testing Connectivity: once you can prove packets reach a host, DNS is the next layer that decides which host you were even trying to reach. Recall the simplified troubleshooting stack we use throughout this series:

Application → TLS → Port → DNS → Gateway → Route → Interface

DNS sits right in the middle. When the layers below it are healthy but the application still fails, DNS is the first place a disciplined engineer looks.

What You Will Learn

• The DNS record types a DevOps engineer works with — A, AAAA, CNAME, MX, TXT, NS, PTR, SOA — and what each one answers.
• How to read dig output and choose between dig, dig +short, per-type queries, reverse lookups, and dig +trace.
• How to use resolvectl to see which resolver and search domains your system is really using.
• The resolution chain from application to authoritative server, and how to isolate which link is broken.
• Concrete symptom → command → conclusion patterns for wrong records, stale caches, bad resolvers, timeouts, split DNS, search-domain surprises, and container DNS.

How Resolution Actually Works

Before running commands, hold the shape of the system in your head. When an application resolves a name, the request travels through a chain:

Application
   │  getaddrinfo("db.internal")
   ▼
Stub Resolver (libc / systemd-resolved)
   │  reads /etc/resolv.conf, adds search domains
   ▼
Recursive DNS (your configured resolver:
   1.1.1.1, corporate DNS, 127.0.0.11 in Docker)
   │  caches answers, does the legwork
   ▼
Authoritative DNS (the name servers that
   actually hold the zone's records)

Two roles matter more than any other, and confusing them causes most DNS confusion. The recursive resolver does the lookup work on your behalf and caches the result — it holds a copy that can go stale. The authoritative server is the source of truth that actually holds the records. Almost every “it works here but not there” DNS incident comes down to two resolvers holding two different cached views of the same name.

The stub resolver on your host is the piece that decides which recursive resolver to ask in the first place. On Kali and most modern Debian/Ubuntu systems that’s systemd-resolved, and getting a straight answer about what it is doing is exactly what resolvectl is for.

🛠️ DevOps Perspective — The single most useful DNS habit: resolve the failing name from the same host and same resolver the application uses, not from your laptop. A name that resolves perfectly on your workstation can return NXDOMAIN inside a container or a cloud VM because they point at different recursive resolvers. “Works on my machine” is often literally a DNS statement.

DNS Record Types You’ll Meet

You don’t need the full zoo of record types, but this handful shows up constantly in infrastructure work. Each one answers a specific question.

Record	What it maps	Answers the question
A	Name → IPv4 address	Where does this name point (IPv4)?
AAAA	Name → IPv6 address	Where does this name point (IPv6)?
CNAME	Name → another name (alias)	What is this name really an alias for?
MX	Domain → mail server names	Where does email for this domain go?
TXT	Name → arbitrary text	SPF/DKIM/DMARC and domain-verification strings
NS	Domain → authoritative name servers	Who holds the real records for this zone?
PTR	IP → name (reverse)	What name claims this IP address?
SOA	Zone metadata + serial number	Who is authoritative, and what’s the zone version?

A few definitions worth nailing down, because troubleshooting hinges on them:

• A and AAAA are the workhorses — the actual addresses a client connects to. A name can have several of each; a load balancer returning multiple A records is normal DNS-level distribution.
• A CNAME is a pure alias: “this name is really that other name.” It cannot coexist with other records at the same name, which is why you can’t put one on a bare/apex domain — that’s what “CNAME at apex” errors mean. Cloud load balancers and CDNs are almost always reached through a CNAME.
• NS records are the delegation: they name the authoritative servers for a zone. Following a name from the root down means following NS records.
• PTR is the reverse of A — an IP mapped back to a name, stored in the in-addr.arpa tree. It matters most for mail deliverability and readable logs.
• SOA (“Start of Authority”) holds zone metadata, most usefully the serial number. If two name servers report different SOA serials, the zone hasn’t finished syncing between them.

Asking DNS Precise Questions With dig

dig (Domain Information Groper) is the DevOps default because it’s precise, scriptable, and shows you everything: the answer, its TTL, which server responded, and the query status. Learn to read it once and you’ll never guess about DNS again.

The full query

dig example.com

This asks your default resolver for example.com (an A record, by default) and prints labeled sections. The two that matter most:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4821
;; ANSWER SECTION:
example.com.        3600    IN    A    93.184.216.34

;; SERVER: 127.0.0.53#53(127.0.0.53)

Read the answer line left to right: the name, the TTL in seconds (3600 — how long a resolver may cache this), the class (IN), the type (A), and the value. Then read two things above and below it. The header status tells you the outcome: NOERROR means the query succeeded (even if there’s no answer of that type), NXDOMAIN means the name genuinely doesn’t exist, and SERVFAIL means the resolver couldn’t complete the lookup — often DNSSEC or an upstream problem, not a missing record. The SERVER line at the bottom tells you which resolver actually answered — here 127.0.0.53, the systemd-resolved stub. That one line prevents a lot of wasted time, because it confirms you’re asking who you think you’re asking.

Reach for the full dig when you need the whole story: the status code, the TTL that explains a stale answer, and the responding server.

dig +short — just the value

dig +short example.com
# 93.184.216.34

+short strips everything except the answer values. This is what you pipe into health checks, CI assertions, and deploy scripts — for example, confirming a hostname points at the address you expect before flipping traffic. Reach for it when you want a clean value, not a diagnosis.

Querying a specific record type

You can ask for any record type by naming it after the domain:

dig example.com A        # IPv4 address
dig example.com AAAA     # IPv6 address
dig example.com TXT      # SPF/DKIM/DMARC and verification strings

dig example.com A and dig example.com AAAA are how you tell an IPv4 problem from an IPv6 one — a name that resolves for A but not AAAA (or vice versa) will fail only for clients using that protocol, which produces baffling “works for some users” reports. dig example.com TXT prints the exact bytes of TXT records, which is precisely what an email or domain-verification checker sees; a verification that “won’t validate” is usually a typo, extra quotes, or the record placed at the wrong name. Reach for typed queries whenever the kind of record is in question, not just its value.

dig -x — reverse lookups (PTR)

dig -x 192.0.2.10 +short
# host10.example.com.

The -x flag builds the special in-addr.arpa reverse query for you, turning an IP back into a name via its PTR record. Reach for it when a log or a ss output shows an IP you don’t recognize, or when diagnosing mail deliverability — many mail servers reject senders whose IP has no PTR, or whose PTR doesn’t match its forward A record. Empty output means no PTR is published for that address.

dig +trace — follow the delegation from the root

dig +trace example.com

+trace makes dig do the recursive resolver’s job itself: it starts at the root name servers, follows the NS delegation down through .com to the domain’s authoritative servers, and prints every hop. Crucially, it bypasses your local resolver’s cache and asks the authoritative chain directly. Reach for it in two situations. First, when you suspect a delegation problem — if the trace stalls or shows unexpected name servers at a hop, you’ve found where the zone is misconfigured. Second, and more commonly, when you need to know what’s true versus what’s merely cached.

🔎 Troubleshooting Tip — If dig example.com (via your cached resolver) and dig +trace example.com disagree, your resolver is serving a stale cached answer. The records are correct at the source; you’re waiting out a TTL or need to flush that resolver. This authoritative-vs-cached comparison is the most valuable DNS diagnostic you can learn — it tells you whether to fix DNS or just wait.

Seeing Your Real Resolver With resolvectl

dig tells you what an answer is; resolvectl tells you who your system asks and how. On Kali and modern Debian/Ubuntu, systemd-resolved is the stub resolver, and /etc/resolv.conf usually points at its local stub address 127.0.0.53. That means the nameserver line in resolv.conf is often a proxy, not the real upstream — so reading the file alone can mislead you.

resolvectl status

This prints, per link, the actual DNS Servers in use, the DNS Domain search list, and whether DNSSEC is active. Read the global block and each interface block: the resolver your traffic really hits is listed there, which may differ from what a container or VPN advertises. When you’re chasing “why is this host resolving names differently than that one,” this is the ground truth.

resolvectl query db.internal

resolvectl query resolves a name through systemd-resolved exactly the way an application’s getaddrinfo() call would — applying the search domains and per-link routing. It also prints which server answered and whether the result came from cache. That “where available” detail matters: systemd-resolved can route different domains to different servers (split DNS), and resolvectl shows you that policy where it’s in effect. Reach for resolvectl whenever you need to reconcile “dig says one thing, the app says another” — dig talks to a resolver you name; resolvectl talks through the exact path the application uses.

🏭 Why This Matters in Production — On a host with a VPN, a corporate DNS, and a Docker daemon all in play, three different resolvers can be active at once. dig @1.1.1.1 proving a name resolves tells you nothing about whether the application’s resolver can resolve it. resolvectl status on the affected host, and resolvectl query <name>, close that gap in seconds.

DevOps Perspective

The habit that separates fast DNS diagnosis from slow guessing is this: resolve the failing name from the same host and the same resolver the application uses. An engineer who runs dig on their laptop, sees a clean answer, and declares “DNS is fine” has proven nothing about the container, pod, or VM that’s actually failing — those environments often point at a different recursive resolver (Docker’s embedded resolver, Kubernetes cluster DNS, a cloud VPC resolver) with a different cache and a different view of internal zones.

So when an application reports a connection failure, resolve its exact target name in its environment first:

# On the affected host / in the affected container:
resolvectl query db.internal      # or:
dig +short db.internal

Empty or wrong means you’ve found the outage, and it was never an application bug. Correct means DNS is exonerated and you move down the stack to Port and TLS. Either way you’ve turned a vague “the service is down” into a precise statement about one layer.

Try It Yourself

🧪 Try It — Every command below is a read-only lookup against public infrastructure or your own host, so it’s safe to run anywhere. Work through them in order and read the differences between the answers:

dig example.com                    # full output: status, TTL, SERVER
dig +short example.com             # just the address
dig example.com AAAA               # the IPv6 answer (or none)
dig example.com TXT                # SPF / verification strings
dig example.com NS +short          # the authoritative name servers
dig +trace example.com             # walk root → .com → authoritative
dig -x 93.184.216.34 +short        # reverse lookup (PTR)
resolvectl status                  # which resolver am I really using?
resolvectl query example.com       # resolve the app's way

Compare the plain dig example.com answer against dig +trace example.com: when they match, DNS is healthy and cached correctly; when they differ, you’ve isolated a caching or delegation problem. Then compare the SERVER line in dig against the DNS Servers reported by resolvectl status — that tells you whether the tool and your applications even agree on who the resolver is.

Common Problems

Each scenario below follows the same discipline: a symptom, the command that reveals the cause, and what you should conclude. Never stop at “it’s DNS” — identify which DNS problem it is.

Wrong A record

Symptom: the app connects, but to the wrong server (old host, wrong region).

dig +short app.example.com A
dig +short @a.iana-servers.net app.example.com A   # ask an authoritative NS

If the resolver returns one address and the authoritative server returns another, a cache is stale (see below). If the authoritative server itself returns the wrong address, the record was simply never updated at the source — this is a DNS change that didn’t happen, not a caching artifact.

Wrong or unexpected CNAME

Symptom: a name points somewhere surprising, or TLS breaks because the certificate doesn’t match the real target.

dig app.example.com
# app.example.com.  300  IN  CNAME  lb-1234.us-east-1.elb.amazonaws.com.
# lb-1234...        60   IN  A      203.0.113.5

The full dig output shows the CNAME chain and its final A records. Conclude by checking the target: a CNAME pointing at a decommissioned load balancer or the wrong provider hostname explains connection and certificate failures that look like application bugs.

Stale DNS (TTL / caching)

Symptom: you updated a record, but some clients still get the old value.

dig app.example.com A         # note the TTL in the ANSWER SECTION
dig +trace app.example.com    # what the authoritative chain says now

If +trace shows the new value but your cached resolver shows the old one, the resolver is inside the old TTL window — a caching issue, not a record error. Read the TTL to know how many seconds the old answer can survive. The fix is to wait out the TTL or flush the resolver, not to keep re-editing the zone.

🔎 Troubleshooting Tip — Watch a cache expire live: run dig app.example.com A repeatedly and watch the TTL count down on each call. When it hits zero and resets, that resolver has re-fetched — for that resolver, your change is now live.

Incorrect resolver (/etc/resolv.conf)

Symptom: one host resolves internal names, an otherwise-identical host doesn’t.

cat /etc/resolv.conf          # what nameserver is configured?
resolvectl status             # what resolver is REALLY in use?

If /etc/resolv.conf points at 127.0.0.53, the real upstream is whatever resolvectl status reports per link — check that. If it points directly at a public resolver like 8.8.8.8, that host cannot resolve internal-only names at all, because a public resolver has no idea what db.internal is. Conclude by comparing the working and broken hosts’ resolvers side by side; the difference is the cause.

DNS timeout

Symptom: lookups hang for seconds, then fail; the app throws slow timeouts, not instant errors.

dig app.example.com
# ;; connection timed out; no servers could be reached

A timeout (as opposed to NXDOMAIN or SERVFAIL) means the resolver itself is unreachable — a firewall dropping UDP/TCP port 53, a downed resolver, or a wrong resolver address. This mirrors the connection timeout signature from Testing Connectivity: silence, not rejection, points at a drop or an unreachable server. Test a known-good resolver to confirm: dig @1.1.1.1 example.com — if that works and your configured resolver times out, the resolver or the path to it is the problem.

Split DNS (internal vs external answers differ)

Symptom: a name resolves to a private address inside the network and a public one outside — or fails entirely from one side.

dig +short app.example.com                 # via your configured resolver
dig +short @1.1.1.1 app.example.com        # via a public resolver

If the internal resolver returns 10.x/192.168.x and the public resolver returns a different (or empty) answer, that’s split-horizon DNS working as designed: internal clients are meant to get the private view. It becomes a bug when a host is on the wrong side — e.g. a container using a public resolver that can’t see the internal view. Conclude by matching each host to the resolver it should be using with resolvectl status.

Search-domain problems (resolv.conf search list)

Symptom: short names like api resolve on one host and not another, or resolve to the wrong environment.

resolvectl status | grep -i "DNS Domain"   # the search list in effect
dig +short api                             # bare name — search domains apply
dig +short api.prod.example.com            # fully qualified — no search magic

The search domain list is what turns a bare api into a fully qualified name by appending suffixes (api.prod.example.com, then api.example.com, and so on). If a host’s search list points at staging.example.com instead of prod.example.com, the same short name silently resolves to the wrong environment. Conclude by always testing the fully qualified name to remove search-list ambiguity — if the FQDN resolves correctly but the short name doesn’t, the search list is the culprit.

Container DNS issues (Docker and Kubernetes)

Symptom: a name resolves fine on the host but fails inside a container or pod.

Inside a Docker container on a user-defined network, /etc/resolv.conf points at Docker’s embedded DNS resolver at 127.0.0.11. That resolver provides service discovery by container name and forwards everything else upstream:

# Inside the container:
cat /etc/resolv.conf          # expect: nameserver 127.0.0.11
dig +short web                # resolve another container by its service name
dig +short example.com        # forwarded to the host's upstream resolver

If dig web fails, the two containers likely aren’t on the same user-defined network (the default bridge has no name-based discovery). If dig web works but dig example.com fails, the embedded resolver can’t reach its upstream — a host-level DNS or firewall problem. This connects directly to the Docker networking and Docker DNS troubleshooting lessons in the Kali-on-Docker series.

In Kubernetes, pods are configured to use the cluster DNS service (CoreDNS), usually at a fixed ClusterIP, with a search list that lets short service names resolve within a namespace:

# Inside a pod (or a debug container):
cat /etc/resolv.conf          # nameserver = cluster DNS; note the search list
dig +short my-svc.my-ns.svc.cluster.local

If the fully qualified svc.cluster.local name fails, suspect CoreDNS itself or a NetworkPolicy blocking port 53 to it; if only the short name fails, the pod’s search domains are wrong. A future “Kali Linux + Kubernetes” path in this series will go deeper — for now, the discipline is identical: resolve the failing name from inside the pod, using the FQDN to remove search-list ambiguity.

🔐 Security Note — Querying public DNS is a benign, everyday read-only operation and is safe from any machine. Probing infrastructure DNS — internal resolvers, zone transfers, reverse-sweeping address ranges — is different: only test systems you own or are explicitly authorized to assess. Keep experiments to controlled labs and your own hosts, and frame this work as authorized infrastructure validation, not reconnaissance.

Troubleshooting Workflow

When a name “doesn’t work,” walk this order instead of guessing:

1. Resolve it the app’s way. On the affected host/container: resolvectl query <name> or dig +short <name>. Empty or wrong here means DNS owns the outage.
2. Read the status. NXDOMAIN = name doesn’t exist; SERVFAIL = resolver couldn’t complete (often DNSSEC/upstream); NOERROR with no answer = the name exists but not for that record type; timeout = the resolver is unreachable.
3. Confirm the resolver. resolvectl status and cat /etc/resolv.conf — is the host even asking the right server?
4. Compare cached vs authoritative. dig <name> versus dig +trace <name> (or dig @<NS> <name>). Disagreement = a caching/TTL issue, not a bad record.
5. Remove search-list ambiguity. Test the fully qualified name; if the FQDN works but the short name doesn’t, the search domains are wrong.
6. Right record type, right family? Check A vs AAAA and the exact TXT/CNAME bytes — protocol- and type-specific bugs hide here.

Only after this do you touch a zone file — and record changes carry real risk, so make them only on domains you administer, one change at a time, mindful of the current TTL.

What You Learned

• DNS resolution is a chain — Application → stub resolver → recursive resolver → authoritative server — and most “outages” are a recursive resolver serving a stale or wrong cached answer, not a broken record at the source.
• The record types a DevOps engineer lives with — A, AAAA, CNAME, MX, TXT, NS, PTR, SOA — each answer a specific operational question, and every record’s TTL controls how long it can be cached.
• dig asks precise questions: full output for status/TTL/SERVER, +short for scripts, per-type queries (A/AAAA/TXT) to isolate protocol and record bugs, -x for reverse PTR lookups, and +trace to follow delegation and reveal what’s true versus merely cached.
• resolvectl reveals your real resolver and search domains on systemd-resolved systems, closing the gap between what dig proves and what your applications actually experience.
• Failures have signatures — wrong A/CNAME, stale TTL caches, a wrong resolver in resolv.conf, timeouts (resolver unreachable), split-horizon differences, search-domain surprises, and container/cluster DNS quirks — and a fixed workflow, always run from the affected host, tells them apart.

Names now resolve — or you know exactly why they don’t. Next, when a name resolves but traffic still doesn’t arrive, you need to see where along the path it’s being lost. Continue to Part 10 — Traceroute, MTR, and Path Analysis, where we trace the route packets take hop by hop and pinpoint where they disappear.