HTTP and API Troubleshooting With Kali Linux

When a web service or API misbehaves, a browser tells you that something is wrong but rarely where. The command line is far more precise: you can see the exact status code, the response headers, the redirect chain, the TLS handshake, and how long each phase took. This lesson shows how to troubleshoot HTTP and API problems with four tools Kali ships by default — curl, wget, openssl, and jq — using benign public endpoints so you can practice safely.

If you have not yet worked through the lower layers, it helps to read networking fundamentals and DNS troubleshooting first, because an “HTTP problem” is often really a DNS or TLS problem in disguise.

The Troubleshooting Model: Layer by Layer

A single HTTP request quietly passes through several layers. When a request fails, your job is to figure out which layer broke. Work from the bottom up:

Browser/API Client
        ↓
DNS
        ↓
TCP
        ↓
TLS
        ↓
HTTP
        ↓
Application

Each layer depends on the ones below it. If DNS does not resolve, TCP never connects. If TCP does not connect, TLS never negotiates. If TLS fails, no HTTP request is sent. And if HTTP returns a 500, the transport was fine — the problem is up in the application. The tools below let you observe each layer independently so you can stop guessing.

🔎 Troubleshooting Tip — Read the status code and the timing together to localize the failure. A connection that never returns a status code failed below HTTP (DNS, TCP, or TLS). A fast 4xx is usually a client/request problem; a slow 5xx is usually the application or a backend it depends on. The number and the clock tell you which layer to inspect next.

curl: Your Primary HTTP Tool

curl is the workhorse. It speaks HTTP (and many other protocols), prints exactly what the server sent, and exposes every phase of the request. Kali installs it by default.

A Simple GET Request

# Fetch a page and print the body to your terminal
curl https://example.com/

By default curl prints only the response body. For troubleshooting you almost always want more than that — the headers and the status code — which the flags below provide.

# -s  silent (hide the progress meter)
# -S  still show errors even when silent
# -o /dev/null  throw away the body, keep the diagnostics
curl -sS -o /dev/null https://example.com/

Response Codes and Headers

The status code is the single most important signal in HTTP troubleshooting. Use -I to send a HEAD request and see only the response headers, or -i to include headers alongside a normal GET.

# HEAD request: headers only, no body — quick and cheap
curl -I https://example.com/

HTTP/2 200
content-type: text/html; charset=UTF-8
cache-control: max-age=604800
content-length: 1256

The first line carries the status code. Memorize the families rather than individual numbers:

Range	Meaning	Typical cause
2xx	Success	The request worked (200 OK, 201 Created, 204 No Content)
3xx	Redirect	Resource moved (301, 302, 307, 308) — see redirects below
4xx	Client error	Your request is wrong (400, 401, 403, 404, 429)
5xx	Server error	The application or a backend failed (500, 502, 503, 504)

A quick rule of thumb: 4xx means fix the request; 5xx means the server (or something behind it) is broken. A 502/504 from a proxy usually means the upstream application is down or too slow, not that the proxy itself is misconfigured.

🛠️ DevOps Perspective — In a pipeline, the exit criteria for a deploy smoke test is almost always “does this endpoint return 2xx?” curl -sS -o /dev/null -w '%{http_code}' gives you exactly that number with no HTML to parse, which is why it shows up in so many health-check scripts and readiness probes.

POST Requests and Sending Data

To exercise an API you often need to send data, not just read it. Use -X to set the method and -d (or --data) for a body. Set the Content-Type header so the server parses it correctly.

# POST a JSON body to a request-inspection endpoint that echoes it back
curl -sS -X POST https://httpbin.org/post \
  -H 'Content-Type: application/json' \
  -d '{"service":"checkout","replicas":3}'

https://httpbin.org is a public request-and-response inspection service — it reflects whatever you send back to you, which makes it ideal for practicing requests without touching any real system. The response includes your headers, body, and the server’s view of the request, so you can confirm you are sending what you think you are.

💡 Note — For form-encoded data use -d 'key=value&key2=value2' with Content-Type: application/x-www-form-urlencoded (curl’s default for -d). For JSON APIs, always set -H 'Content-Type: application/json' explicitly, or the server may reject or misread the body.

Following Redirects With -L

By default curl does not follow redirects — it shows you the 3xx and stops. That is useful for debugging (you see the exact hop), but when you want the final page, add -L to follow the Location header.

# Without -L: see the redirect itself
curl -I https://httpbin.org/redirect-to?url=https://example.com/

# With -L: follow the chain to the final destination
curl -sSL -o /dev/null -w '%{http_code} %{url_effective}\n' \
  'https://httpbin.org/redirect-to?url=https://example.com/'

%{url_effective} prints where you actually ended up. Redirect loops, unexpected downgrades from HTTPS to HTTP, or a login page appearing mid-chain are all common bugs that only surface when you inspect each hop.

🔎 Troubleshooting Tip — If a page “works in the browser but fails in a script,” suspect redirects. Browsers follow them automatically; a bare curl or health check does not. Reproduce with curl -IL <url> to see the full chain, then decide whether your client should follow it or target the final URL directly.

Timing a Request

curl’s -w (write-out) flag prints timing variables after the transfer, breaking the request into phases. This is how you tell a slow DNS lookup apart from a slow application.

curl -sS -o /dev/null -w '\
dns:     %{time_namelookup}s\n\
connect: %{time_connect}s\n\
tls:     %{time_appconnect}s\n\
ttfb:    %{time_starttransfer}s\n\
total:   %{time_total}s\n' https://example.com/

Each variable is a cumulative timestamp measured from the start of the request:

Variable	Marks the end of	If it dominates, look at
time_namelookup	DNS resolution	DNS / resolver (DNS troubleshooting)
time_connect	TCP handshake	Network path, firewall, distance
time_appconnect	TLS handshake	Certificates, cipher negotiation (TLS troubleshooting)
time_starttransfer	First response byte (TTFB)	The application generating the response
time_total	Entire transfer	Overall, plus body size / bandwidth

Because the values are cumulative, the gaps between them localize the slowness. A large jump between time_appconnect and time_starttransfer means TLS was fine but the application was slow to respond — a backend or database problem, not a network one.

🛠️ DevOps Perspective — This one-liner is the fastest way to answer “is it the network or the app?” during an incident. Because it maps directly onto the layer model, the biggest gap in the output points straight at the layer to investigate — no packet capture required.

Verbose Mode: -v

When you need to see everything — the DNS result, the connection, the TLS handshake summary, and the full request and response headers — use -v.

# Verbose: shows connection, TLS, and header exchange
curl -v -o /dev/null https://example.com/

In the verbose output, lines prefixed with * are curl’s own diagnostics (connection and TLS), > lines are the request headers curl sent, and < lines are the response headers the server returned. This single view often reveals the problem instantly: a wrong Host header, an unexpected redirect, a missing cookie, or a TLS negotiation that never completes.

wget: Fetching and Mirroring

wget overlaps with curl but shines at downloading files and following links non-interactively — useful in scripts and for retrieving artifacts on a box with no browser.

# Download a file, saving it under its remote name
wget https://example.com/index.html

# Quiet download to a specific path, following redirects (wget follows by default)
wget -q -O /tmp/page.html https://example.com/

# Show only the response headers without saving the body
wget -q -S --spider https://example.com/

--spider makes wget check that a URL exists without downloading it — handy for validating that a release artifact or endpoint is reachable before a deploy step depends on it. As a rule, reach for curl when you are inspecting an API and wget when you are retrieving a file.

jq: Parsing JSON APIs

Modern APIs speak JSON, and raw JSON on a terminal is hard to read. jq is a command-line JSON processor: it pretty-prints, filters, and extracts fields so you can pull exactly the value you need out of an API response.

# Pretty-print a JSON response
curl -sS https://httpbin.org/json | jq .

# Extract a single nested field
curl -sS https://httpbin.org/json | jq '.slideshow.title'

# Pull specific fields into a compact object
curl -sS https://httpbin.org/get | jq '{url: .url, agent: .headers["User-Agent"]}'

jq . reformats and colorizes the whole document. A path like .slideshow.title drills into nested objects; .[] iterates arrays; and you can build new objects with { ... }. In troubleshooting, this lets you confirm an API returned the field your application expects — a null or a missing key here often explains a downstream failure.

🧪 Try It — Chain curl and jq to inspect exactly what a server sees when you call it. Run this and read the origin and headers fields it returns:

curl -sS https://httpbin.org/get | jq '.headers'

Then add a custom header and confirm it round-trips: curl -sS -H 'X-Trace-Id: abc123' https://httpbin.org/get | jq '.headers["X-Trace-Id"]'. This is how you verify a proxy, gateway, or client is actually sending the headers you configured.

API Health Endpoints

Most services expose a health or readiness endpoint (commonly /health, /healthz, /status, or /livez) that returns a small JSON document and a 2xx when the service is up. Checking it is the first move when an API “feels down.”

# Check a health endpoint: status code + latency, parse the body if JSON
curl -sS -o /dev/null -w '%{http_code} %{time_total}s\n' https://example.com/healthz

# When the endpoint returns JSON, inspect the reported state
curl -sS https://example.com/healthz | jq '.status'

A good health check distinguishes “the process is alive” (liveness) from “it can serve traffic” (readiness). If liveness passes but readiness fails, the process is running but a dependency — a database, cache, or downstream API — is unreachable. That is exactly the kind of thing the timing breakdown and jq on the health body help you pin down.

🛠️ DevOps Perspective — Kubernetes liveness and readiness probes are just HTTP checks against these endpoints. Reproducing a failing probe by hand with the same curl the kubelet would run is often the quickest way to understand why a pod is restarting or stuck out of the load-balancer rotation.

Authentication Headers (Conceptually)

APIs that require authentication expect a credential, usually in a header — most often Authorization. The two common shapes are a bearer token and HTTP basic auth. The mechanics look like this, with placeholders standing in for real credentials:

# Bearer token (the value is a placeholder — never a real token)
curl -sS -H 'Authorization: Bearer <YOUR_TOKEN>' https://example.com/api/v1/status

# Basic auth (curl builds the Authorization header for you)
curl -sS -u '<username>:<password>' https://example.com/api/v1/status

When authentication is the suspect, the status code narrows it down fast: 401 Unauthorized means the credential was missing or invalid; 403 Forbidden means you authenticated but lack permission for that resource. Those are different fixes — a bad token versus a missing role.

🔐 Security Note — Never paste real tokens, passwords, or API keys onto the command line: they land in your shell history, in process listings (ps shows other users the full command), and often in logs. Read secrets from an environment variable or a file instead, e.g. -H "Authorization: Bearer $API_TOKEN", and scope tokens to the minimum they need. Only test systems you own or are explicitly authorized to test.

TLS Inspection With curl and openssl

When the failure is at the TLS layer — an expired certificate, a hostname mismatch, or a broken chain — you need to look at the handshake directly. This deserves its own lesson (TLS certificate troubleshooting); here are the essentials for HTTP work.

Testing Before DNS Is Updated: —resolve

curl --resolve lets you override DNS for a single request, sending the connection to a specific IP while still presenting the real hostname (so TLS and virtual-host routing behave correctly). This is how you test a new server before you cut DNS over to it.

# Force example.com to resolve to a specific IP for this request only
curl -sS -o /dev/null -w '%{http_code}\n' \
  --resolve example.com:443:203.0.113.10 https://example.com/

Because the hostname in the URL stays example.com, the server still receives the correct SNI and Host header — unlike hard-coding the IP in the URL, which would break TLS validation and name-based routing.

Inspecting the Certificate Directly

openssl s_client opens a raw TLS connection and shows the certificate the server presents, including who issued it and when it expires.

# Show the certificate's validity dates for a host
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -dates -subject -issuer

The -servername flag sends SNI so the server returns the right certificate for name-based virtual hosts. -noout -dates prints notBefore/notAfter — an expired notAfter is one of the most common production TLS outages, and this one line confirms it.

The -k Flag: Understand the Risk

curl -k (or --insecure) tells curl to proceed even if certificate validation fails. It is a diagnostic aid, not a fix.

# -k skips certificate verification — DIAGNOSTIC USE ONLY
curl -k -sS -o /dev/null -w '%{http_code}\n' https://example.com/

⛔ Production Warning — Use -k only to confirm a suspicion during debugging (for example, to prove that the only problem is an untrusted or expired certificate). Never put -k in a script, health check, or client that talks to production: it disables the protection TLS exists to provide and turns any interception into a silent success. If -k makes a request work, the correct fix is to repair the certificate or trust chain, not to keep skipping verification.

A Complete Troubleshooting Workflow

Put the layers together. When an endpoint is failing, walk up the stack and stop at the first layer that breaks:

# 1. DNS — does the name resolve at all?
curl -sS -o /dev/null -w 'dns_time: %{time_namelookup}s\n' https://example.com/

# 2. TCP + TLS — do connect and handshake complete?
curl -sS -o /dev/null -w 'connect: %{time_connect}s tls: %{time_appconnect}s\n' https://example.com/

# 3. HTTP — what status code, and how fast to first byte?
curl -sS -o /dev/null -w '%{http_code} ttfb: %{time_starttransfer}s\n' https://example.com/

# 4. Application — inspect the body / health state
curl -sS https://example.com/healthz | jq '.'

Read the output against the layer diagram. A DNS time of 0s with everything else failing points at resolution; a completed TLS phase with a slow TTFB points at the application. Each command isolates one band of the stack so the failure has nowhere to hide.

🔎 Troubleshooting Tip — Localize before you dig. If step 1 or 2 fails, the problem is below HTTP — jump to DNS troubleshooting or TLS certificate troubleshooting rather than staring at application logs. Only when the transport is clean and you have a 4xx/5xx does it make sense to read the app’s logs.

🧪 Try It — Run this against a public endpoint and note the two numbers it prints — the status code and the total time:

curl -sS -o /dev/null -w '%{http_code} %{time_total}s\n' https://example.com/

A 200 with a small total time means every layer is healthy. Now try it against https://httpbin.org/status/503 and https://httpbin.org/delay/3 to see what a server error and a slow response look like through the same lens. This tiny command is the seed of most HTTP health checks you will ever write.

What You Learned

• HTTP problems are best diagnosed layer by layer — DNS → TCP → TLS → HTTP → Application — and the goal of every command is to find the first layer that breaks.
• curl is the primary tool: -I/-i for headers and status codes, -d/-X for POST, -L to follow redirects, -v for the full handshake and header exchange, and -w timing variables to tell a slow network apart from a slow app.
• Status code plus timing localizes the failure: no code at all means the transport failed; a fast 4xx is a request problem; a slow 5xx is the application or a backend.
• jq turns raw JSON into readable, filterable output so you can confirm an API returned the fields your application expects, including on /healthz-style endpoints.
• For TLS, curl --resolve tests a target before DNS changes, openssl s_client reveals the certificate’s issuer and expiry, and -k is a diagnostic-only flag that must never live in production clients.
• Keep credentials out of your shell history and process list, and only test systems you own or are explicitly authorized to test.