🚨 Grafana Error Guide: 'Templating [$var] failed to load values' — Fix Broken Variable Queries

Overview

Grafana template variables let a dashboard populate a dropdown (region, namespace, instance) by running a query against a datasource. When that query returns an error, an empty result, or points at the wrong datasource, the variable cannot resolve and Grafana refuses to render dependent panels.

The literal error you see at the top of the dashboard:

Templating [namespace] failed to load values

Error updating options: <underlying datasource error>

For a Prometheus variable the underlying error is usually a bad label_values() expression; for SQL variables it is a syntax error or missing column. Because every panel that references $namespace depends on this variable, one broken variable can blank out the entire dashboard even though the datasource itself is healthy.

Symptoms

• A red “Templating [$var] failed to load values” banner at the top of the dashboard.
• Variable dropdowns are empty or show None.
• Panels using the variable render “No data” or an interpolation error like parse error: unexpected {.
• The dashboard worked before an import, datasource rename, or Grafana upgrade.

Common Root Causes

1. Variable points at the wrong or deleted datasource

After an import or provisioning change, the variable’s datasource UID no longer matches any installed datasource, so the query never runs.

2. Invalid variable query expression

A typo in label_values(metric, label), a misspelled label, or an SQL error causes the datasource to reject the query.

3. The label or metric no longer exists

The metric was renamed or dropped by a relabel rule, so label_values() returns nothing and dependent chained variables cascade to empty.

4. Query timeout or datasource proxy error

A slow label_values() over high-cardinality data exceeds the datasource timeout and surfaces as a load failure.

5. Permissions / row-level security

The dashboard viewer’s account lacks access to the datasource or the underlying rows, so the variable query is denied.

Diagnostic Workflow

Step 1: Read the exact underlying error

Open the variable dropdown, then check the browser devtools Network tab or the Grafana server log for the real datasource error:

sudo journalctl -u grafana-server --no-pager | grep -iE "template|variable|datasource" | tail -20
# Container / Kubernetes
kubectl logs deploy/grafana -n monitoring | grep -iE "template|variable" | tail -20
grep -iE "failed to load values|variable" /var/log/grafana/grafana.log | tail -20

Step 2: Confirm the variable’s datasource still exists

curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  http://localhost:3000/api/datasources | \
  jq '.[] | {name, uid, type}'

Match a returned uid against the datasource set on the variable (Dashboard settings → Variables → your variable → Data source). A ${DS_PROMETHEUS} placeholder that was never remapped on import is a classic culprit.

Step 3: Run the variable query directly

For a Prometheus label_values() variable, test the underlying series call:

count by (namespace) (kube_pod_info)

Then confirm the label actually exists:

curl -s "http://prometheus:9090/api/v1/label/namespace/values" | jq '.data | length'

An empty data array means the label is gone — fix the metric name or relabeling, not Grafana.

Step 4: Check for timeouts on high-cardinality labels

# grafana.ini — raise if variable queries are slow
[dataproxy]
timeout = 30

If the query only fails on large time ranges, scope the variable query with a time constraint or a more specific metric.

Example Root Cause Analysis

A team imports a community dashboard JSON. On load, every panel is blank and the banner reads:

Templating [instance] failed to load values
Error updating options: bad_data: 1:1: parse error: unknown function "labl_values"

The variable definition contains a typo — labl_values(node_uname_info, instance) instead of label_values(...). Because $instance never resolves, all panels that filter by instance="$instance" produce empty PromQL.

Fix: Dashboard settings → Variables → instance → correct the query to label_values(node_uname_info, instance), click “Run query”, confirm values appear, then Save. All panels immediately populate. The root cause was a copy-paste error in the imported JSON, not the Prometheus datasource.

Prevention Best Practices

• Provision variables and datasource UIDs together so imports never leave a ${DS_*} placeholder unresolved; see more Grafana guides.
• Prefer stable label_values(metric, label) forms and pin the metric name to something that survives relabeling.
• Set a sane [dataproxy] timeout and avoid variables over extremely high-cardinality labels; use a narrower metric.
• Add a -- / “All” option only where panels handle an empty selection gracefully.
• Test dashboards in a staging Grafana after every datasource rename or Grafana upgrade.
• Give viewers query access to the variable’s datasource, or the dropdown silently fails to load.

Quick Command Reference

# Server-side error for the failing variable
sudo journalctl -u grafana-server | grep -iE "template|variable" | tail -20
kubectl logs deploy/grafana -n monitoring | grep -i template | tail -20

# List datasource UIDs to match the variable's datasource
curl -s -H "Authorization: Bearer $GRAFANA_TOKEN" \
  http://localhost:3000/api/datasources | jq '.[] | {name, uid, type}'

# Verify a Prometheus label actually has values
curl -s "http://prometheus:9090/api/v1/label/namespace/values" | jq '.data | length'

# Raise variable query timeout (grafana.ini)
# [dataproxy]
# timeout = 30

Conclusion

“Templating [$var] failed to load values” means the variable’s query failed, not that your panels are broken. Work from the underlying datasource error outward:

1. Read the real error from the log or Network tab (typo vs. timeout vs. permission).
2. Confirm the variable’s datasource UID still resolves after import/provisioning.
3. Run the variable query directly and verify the label/metric still exists.
4. Raise the proxy timeout only when high cardinality is the genuine cause.

Fix the variable and every dependent panel repopulates at once, because the failure was always upstream of the panels themselves.