🚨 VictoriaMetrics Error: 'cannot parse metric value' — Cause, Fix, and Troubleshooting Guide

Overview

VictoriaMetrics parses samples in the Prometheus text exposition format, where every metric value must be a plain float (or one of the special tokens +Inf, -Inf, NaN). When a scraped /metrics page or an import payload carries a value the float parser cannot read, the ingest path rejects the sample with:

cannot parse metric value for "node_load1": cannot parse "1,25" as float; unsupported number format

This is a value-parsing failure, not a labels or timestamp problem. The most common trigger is a decimal comma (1,25 instead of 1.25) produced by an exporter running under a non-English locale, but any non-numeric or unit-tagged value in the value field will produce the same error.

Symptoms

• Ingest or scrape of a specific target fails while other targets on the same vmagent/vminsert work fine.
• The error names one metric (for example node_load1) and quotes the offending token.
• The quoted value contains a comma, a unit suffix, or trailing text rather than a bare number.
• Samples from a locale-sensitive exporter go missing while integer-valued metrics from the same process ingest normally.
• promtool check metrics against the exporter output reports invalid values.

Common Root Causes

• A non-numeric value — the value field contains text, a status word, or an empty string instead of a number.
• A locale using a decimal comma — the exporter formats floats as 1,25 because it inherited LC_NUMERIC/LANG from a non-English locale.
• Units appended to the value — a value like 1.25s or 512MB where the exporter should emit a bare float and encode the unit in the metric name.
• An exporter bug emitting invalid floats — malformed scientific notation, doubled signs, or thousands separators in the value.

How to diagnose

Pull the raw /metrics page and grep for the offending metric to see exactly what the exporter emits:

curl -s 'http://exporter.internal:9100/metrics' | grep '^node_load1'

If the value shows a comma or unit, confirm the exporter’s locale:

# Inspect the environment of the exporter process
tr '\0' '\n' < /proc/$(pgrep -f node_exporter)/environ | grep -E 'LANG|LC_'

Validate the whole page before it reaches VictoriaMetrics so you catch every bad line at once:

curl -s 'http://exporter.internal:9100/metrics' | promtool check metrics

Fixes

1. Emit dot-decimal numeric values only. The value field must be a bare float using a period as the decimal separator, for example 1.25, 0, 3e6, or one of +Inf, -Inf, NaN. No commas, no thousands separators.

2. Fix the exporter/locale so values are plain floats. Force a numeric-neutral locale for the exporter process so float formatting stops using a comma:

# systemd unit for the exporter
Environment=LC_ALL=C
Environment=LANG=C

3. Strip units from the value field. Move any unit into the metric name (disk_free_bytes rather than a value like 512MB) and expose only the number.

4. Validate /metrics with promtool before ingest. Add promtool check metrics to the exporter’s CI or a pre-scrape health check so malformed values are caught upstream instead of at ingest.

What to watch out for

• The error aborts only the affected sample or scrape target; a single bad line can silence an otherwise healthy exporter, so do not assume the whole pipeline is down.
• Locale bugs are intermittent — they surface only on hosts where LANG/LC_NUMERIC differs, so the same exporter image can behave differently across nodes.
• NaN, +Inf, and -Inf are valid and must not be rewritten to 0; that would corrupt the data.
• Custom or in-house exporters are the usual culprit; well-maintained upstream exporters already format floats correctly.

Related

• VictoriaMetrics Error Guide: cannot parse MetricsQL
• VictoriaMetrics Error Guide: cannot unmarshal labels
• VictoriaMetrics Error Guide: too old timestamp