🚨 Telegraf Error Guide: 'metric parse error ... invalid number' — Fix Line Protocol Parsing

Overview

Telegraf parses input data into metrics according to each input’s data_format. When it tries to read a value as an InfluxDB line-protocol number and the token is not a valid number, it logs a parse error pointing at the offending column:

2026-07-10T12:00:00Z E! [inputs.exec] Error in plugin: metric parse error: expected field at 1:34: "cpu,host=web01 load=hi 1720612800000000000"

The most common wording explicitly calls out the bad numeric token:

E! [inputs.tail] Error parsing "queue_depth,q=jobs value=12.3.4": invalid number at 1:24

The affected line is dropped; other valid lines from the same input still parse, so you see partial data rather than a total outage.

Symptoms

• Some metrics are missing and journalctl -u telegraf shows metric parse error / invalid number for specific lines.
• The error includes a line:column position and echoes the raw input string.
• Data from a script, log tail, or HTTP endpoint is intermittently dropped whenever a value is malformed.
• Integer fields land as floats (or fail) because the i suffix is missing or misplaced.
• A field that should be a string is being parsed as a number because it is unquoted.

Common Root Causes

• Non-numeric value in a numeric field — a script emits load=hi or value=N/A where a number is expected.
• Malformed number — two decimal points (12.3.4), a trailing unit (45ms), a thousands separator (1,024), or a stray sign.
• Missing integer suffix — line protocol needs i for integers (count=5i); mixing 5 and 5i for the same field across lines causes type conflicts.
• Unquoted string field — string values must be double-quoted (status="ok"); unquoted text is parsed as a number and fails.
• Wrong data_format — feeding JSON or CSV data to the default influx parser, so ordinary text is misread as line protocol.
• Locale-formatted decimals — a comma decimal separator (3,14) from a localized tool.
• Timestamp confusion — a value placed in the timestamp position that is not a valid integer nanosecond timestamp.

Diagnostic Workflow

First capture the exact bytes the input produces and inspect them. For an exec collector, run the command and eyeball the line protocol:

sudo -u telegraf /opt/scripts/queue_depth.py | cat -A | head

Run only the affected input under Telegraf with debug so you see the raw string and the parse position together:

telegraf --config /etc/telegraf/telegraf.conf --test --input-filter exec --debug

Validate a suspect line against the parser in isolation by piping it through a file-based test config:

# /tmp/parse-check.conf
[[inputs.file]]
  files = ["/tmp/sample.lp"]
  data_format = "influx"

[[outputs.file]]
  files = ["stdout"]

printf 'queue_depth,q=jobs value=12.3\n' > /tmp/sample.lp
telegraf --config /tmp/parse-check.conf --test

If the source is JSON or CSV, set the correct parser rather than fighting the line-protocol parser:

[[inputs.exec]]
  commands = ["/opt/scripts/stats.py"]
  data_format = "json_v2"
  [[inputs.exec.json_v2]]
    [[inputs.exec.json_v2.field]]
      path = "queue.depth"
      type = "int"

Correct line protocol quotes strings and suffixes integers:

queue_depth,q=jobs value=12i,status="ok" 1720612800000000000

Example Root Cause Analysis

A tail input on an application log intermittently dropped lines with invalid number at 1:31. The team was using data_format = "influx" with a custom log line that occasionally contained latency_ms=timeout when the upstream call failed. Line protocol cannot represent timeout as a numeric field, so those lines were rejected while numeric lines passed — producing gaps exactly during incidents, when the data mattered most.

The proper fix was two-fold. First, the log producer was changed to emit latency_ms=-1i (a sentinel) instead of the word timeout so the field stayed numeric. Second, a status tag captured the qualitative state:

app_latency,endpoint=/checkout,result=timeout latency_ms=-1i 1720612800000000000
app_latency,endpoint=/checkout,result=ok latency_ms=142i 1720612800000000000

After the producer change, telegraf --test --input-filter tail parsed every line and the gaps disappeared. The lesson: numeric fields must always be numeric — encode qualitative states as tags or sentinel integers, never as words in a value position.

Prevention Best Practices

• Never emit words or units in a numeric field; use tags for categorical data and sentinel numbers for “unknown”.
• Always quote string fields (status="ok") and suffix integers consistently (count=5i) in generated line protocol.
• Match data_format to the actual payload — use json_v2, csv, prometheus, etc. instead of forcing influx.
• Validate generated line protocol with a file input + telegraf --test in CI before wiring it into production.
• Keep field types stable over time; a field that is an integer on one line and a float on another causes downstream conflicts.
• Normalize locale in producers so decimals use . and no thousands separators.

Quick Command Reference

# See the exact bytes a collector emits (including hidden chars)
sudo -u telegraf /opt/scripts/queue_depth.py | cat -A | head

# Run one input with debug to see raw string + parse position
telegraf --config /etc/telegraf/telegraf.conf --test --input-filter exec --debug

# Validate a sample line against the influx parser
printf 'm,tag=a value=12i\n' > /tmp/sample.lp
telegraf --config /tmp/parse-check.conf --test

# Watch parse errors live
journalctl -u telegraf -f | grep -i 'parse error\|invalid number'

Conclusion

metric parse error ... invalid number means an input value could not be read as a line-protocol number — usually a non-numeric token, a malformed decimal, a missing i suffix, an unquoted string, or the wrong data_format. Capture the exact bytes the input emits, replay them through a file input with telegraf --test, and either fix the producer to emit valid numeric fields or switch to the parser that matches the payload. Numeric fields must stay numeric — everything qualitative belongs in tags.