🚨 Prometheus Error Guide: 'Empty query result' / No Data for an Existing Metric

Exact Error Message

This is not a hard error — the query is valid and returns a 200, but the result set is empty. The Graph UI shows the chart area blank; Explore prints a notice; the API returns an empty result array:

Empty query result

No data

{"status":"success","data":{"resultType":"vector","result":[]}}

Because status is success, nothing is “broken” from Prometheus’s point of view. The query simply matched zero series at the evaluation time. The job is to figure out why a metric you believe exists produced no samples.

What the Error Means

A query selects series whose labels match your matchers, then evaluates over the requested time range. An empty result means one of three things: no series matched the label matchers, matching series exist but have no sample within the lookback delta (default 5 minutes) of the query time, or the query time falls outside the range where the data lives.

Crucially, this is a data/PromQL problem, not a datasource or connectivity problem. Prometheus answered successfully; the gap is between what you asked for and what is in the TSDB. (If Grafana itself cannot reach Prometheus you would instead see a gateway/datasource error — that is a different guide, linked below.)

Common Causes

1. Label matcher typo or case mismatch

Labels are case-sensitive and exact. {job="API"} will not match job="api", and {instnace=...} matches nothing:

http_requests_total{job="API"}       # data is job="api"
http_requests_total{instance="web1"} # data is instance="web1:8080"

2. The target is down / the metric was never scraped

If the exporter is unreachable or the scrape job is misconfigured, no series for that metric ever land in the TSDB.

3. Staleness — the series went stale (5m lookback)

An instant query looks back up to the lookback delta (default 5m) for the most recent sample. If a target stopped exporting a series more than 5 minutes ago, Prometheus marks it stale and the bare metric returns empty even though historical data exists.

4. Querying outside the data’s time range

In the Graph UI a range set to “last 1h” when the data only exists yesterday returns nothing. Likewise an instant query time= far from when the series was alive.

5. rate() over too short a range

rate(metric[1m]) needs at least two samples inside the window. With a 1m scrape interval and a [1m] range you frequently get zero or one sample per series, so rate() returns empty for those series.

6. A recording rule hasn’t evaluated yet

Querying a recording-rule metric (e.g. job:http_requests:rate5m) right after defining it returns empty until the rule group runs for the first time.

7. The instance (or other) label changed

A pod restart, IP change, or relabel tweak changes instance/pod, so a matcher pinned to the old value matches nothing even though the metric is flowing under a new label value.

How to Reproduce the Error

Match on a label value that does not exist:

curl -s 'http://localhost:9090/api/v1/query' \
  --data-urlencode 'query=up{job="does-not-exist"}' | jq '.data.result'

[]

Or wrap a slow-scraped metric in too short a range:

curl -s 'http://localhost:9090/api/v1/query' \
  --data-urlencode 'query=rate(rarely_scraped_metric[30s])' | jq '.data.result | length'

0

Diagnostic Commands

Strip back to the bare metric name — drop every label matcher. If this returns data, your matchers are the problem; if it is still empty, the metric itself is absent or stale:

curl -s 'http://localhost:9090/api/v1/query' \
  --data-urlencode 'query=http_requests_total' | jq '.data.result | length'

Confirm the metric name even exists in the TSDB:

curl -s 'http://localhost:9090/api/v1/label/__name__/values' \
  | jq -r '.data[]' | grep http_requests_total

List the real label values so you can spot typos/case (here, the actual job values):

curl -s 'http://localhost:9090/api/v1/series' \
  --data-urlencode 'match[]=http_requests_total' \
  | jq -r '.data[].job' | sort -u

Count matching series at evaluation time — count() over your full expression tells you if anything matches:

curl -s 'http://localhost:9090/api/v1/query' \
  --data-urlencode 'query=count(http_requests_total{job="api"})' | jq '.data.result'

Check the target’s health and last scrape — a down target or an old lastScrape explains staleness:

curl -s 'http://localhost:9090/api/v1/targets' \
  | jq -r '.data.activeTargets[] | [.labels.job,.health,.lastScrape] | @tsv'

api     up     2026-06-27T14:31:02.118Z
batch   down   2026-06-27T13:58:11.004Z

Look across history with query_range to find when the data existed, rather than just at “now”:

curl -s 'http://localhost:9090/api/v1/query_range' \
  --data-urlencode 'query=http_requests_total{job="api"}' \
  --data-urlencode 'start=2026-06-27T00:00:00Z' \
  --data-urlencode 'end=2026-06-27T14:00:00Z' \
  --data-urlencode 'step=300s' | jq '.data.result | length'

Step-by-Step Resolution

1. Does the bare metric return data? Query just http_requests_total. If yes, the metric exists — your matchers, time range, or rate() window are wrong. If no, the metric is missing or stale; go to step 4.

2. Fix the label matchers. Use /api/v1/series to read the real label values, then correct case and spelling:

   http_requests_total{job="api", instance="web1:8080"}

3. Widen or move the time range. In the Graph UI pick the period where query_range showed data; for instant queries set time= inside the live window.

4. Check the target. If /api/v1/targets shows the job down, fix scrape connectivity/config — that is why no series exist. If it is up but the series is stale (last sample > 5m ago because the app stopped emitting it), the empty instant result is expected; use query_range over the period it was alive.

5. Lengthen the rate() window to at least 4x the scrape interval so the window always contains multiple samples:

   rate(http_requests_total[5m])

6. For recording rules, confirm the rule group has evaluated:

   curl -s http://localhost:9090/api/v1/rules \
     | jq -r '.data.groups[].rules[] | select(.name=="job:http_requests:rate5m") | .lastEvaluation'

7. Re-run the corrected query in Explore and confirm a non-empty vector.

Prevention and Best Practices

• Build queries by starting from the bare metric name in Explore, then add one matcher at a time so you immediately see which matcher empties the result.
• Use the autocomplete/metric explorer in the Graph UI rather than typing label values from memory — it eliminates case/typo mistakes.
• Set rate()/increase() ranges to at least 4x your scrape interval (commonly [5m] for a 1m scrape) so windows always span multiple samples.
• Alert on target health (up == 0) so a stopped scrape surfaces as an alert instead of as silently empty dashboards.
• Remember the 5-minute lookback: a metric a service only emits intermittently will look “missing” between emissions; consider last_over_time(metric[1h]) to carry the last value forward where appropriate.
• The free incident assistant can diff your query’s matchers against the live label values and point at the exact typo or staleness; see more under Prometheus and monitoring.

Related Errors

• Prometheus Error Guide: Grafana ‘Bad Gateway’ / ‘No data’ — a datasource/gateway problem where Grafana cannot reach Prometheus at all; distinct from this PromQL/data problem where Prometheus answers successfully with an empty set.
• Prometheus Error Guide: ‘parse error: unexpected’ — when the query is malformed rather than merely empty.
• Prometheus Error Guide: ‘out of order sample’ — silent sample drops that can create gaps which look like empty results.

Frequently Asked Questions

The metric shows in the dropdown but my query is empty. Why? The dropdown lists every name ever ingested, including stale ones. The series may have gone stale (no sample within the 5m lookback) or your label matchers may not match the live label values. Query the bare name and check /api/v1/targets for staleness.

Why does rate(metric[1m]) return nothing? rate() needs at least two samples inside the window. With a 1m scrape interval a [1m] window often contains only one sample, so rate() yields no value. Use [5m] or wider.

My instant query is empty but the graph shows data. What gives? The instant query evaluates at a single point (default “now”), and “now” may be past the series’ last sample plus the 5m lookback. The range graph spans a window that still includes live data. Use query_range or move the instant time= into the live window.

How do I see the real label values to fix a typo? Hit /api/v1/series?match[]=<metric> and read the labels, or /api/v1/label/<name>/values for one label’s distinct values. Labels are case-sensitive, so Job ≠ job.

Is an empty result ever the correct answer? Yes. If no series currently satisfy your matchers — the condition genuinely isn’t happening, or the target legitimately isn’t running — an empty vector is correct. Confirm with count() and target health before assuming a bug.