🚨 VictoriaMetrics Error: 'cannot load stream aggregation config: unexpected output' — Cause, Fix, and Troubleshooting Guide

Overview

Stream aggregation lets vmagent (or single-node victoria-metrics) aggregate samples in flight before they are stored, driven by a YAML config passed via -streamAggr.config or -remoteWrite.streamAggr.config. VictoriaMetrics parses and validates that file at startup (and on reload). If any rule is malformed, it refuses to start rather than silently ignoring the rule:

cannot load stream aggregation config from "/etc/vmagent/streamaggr.yml": cannot parse aggregation config: unexpected `output` "totalx"; supported outputs: total, increase, count_samples, sum_samples, last, min, max, avg, quantiles

The error is precise: it names the file, the bad field, and the allowed values. Here totalx is not a real output, so parsing fails. The same fail-fast validation applies to bad YAML, a missing interval, or an invalid match selector — the config must be fully valid before aggregation runs.

Symptoms

• vmagent (or single-node) exits at startup with cannot load stream aggregation config.
• The log lists the supported outputs set, indicating an unknown or misspelled output.
• A config edit followed by a reload aborts and the old config keeps running (or the process crashes).
• Aggregated series stop appearing because the rules never loaded.
• YAML parse errors reference a specific line or field in the aggregation file.

Common Root Causes

• YAML syntax error — bad indentation, a tab, or an unquoted special character breaks parsing.
• Unknown or misspelled output — a value outside total, increase, count_samples, sum_samples, last, min, max, avg, quantiles.
• Missing or invalid interval — no interval set, or a value without a valid duration unit.
• Bad match[] selector — an invalid MetricsQL selector in the rule’s match.
• Wrong by/without usage — grouping keys that conflict or are malformed.

How to diagnose

Validate the YAML shape first, then confirm which flag and file are actually in use:

# Is the YAML even well-formed?
python3 -c 'import yaml,sys; yaml.safe_load(open("/etc/vmagent/streamaggr.yml")); print("yaml ok")'
# Which stream-aggr flag/path is running?
ps aux | grep vmagent | grep -oE '\-(remoteWrite\.)?streamAggr.config[^ ]*'

Inspect the offending rule and check the output, interval, and match fields against the supported set:

grep -nE 'output|interval|match|by|without' /etc/vmagent/streamaggr.yml

Fixes

1. Validate the YAML and correct indentation/tabs so the file parses cleanly (see the diagnose step above).

2. Use only supported outputs. Replace the invalid value with a real one from the list:

- match: 'app_request_duration_seconds_total'
  interval: 30s
  outputs: [total]        # not "totalx"

3. Set a valid interval with a duration unit:

- match: 'http_requests_total'
  interval: 30s
  outputs: [increase]

4. Fix the match[] MetricsQL selector so it is a valid series selector:

- match: '{__name__=~"node_network_.+_bytes_total"}'
  interval: 1m
  outputs: [total]

5. Confirm the flag path points at the file you are editing, then reload so the new config takes effect:

./vmagent -remoteWrite.url=http://vminsert:8480/insert/0/prometheus/ \
  -remoteWrite.streamAggr.config=/etc/vmagent/streamaggr.yml
# Reload after editing:
kill -HUP "$(pidof vmagent)"

What to watch out for

• vmagent fails fast on a bad aggregation config; a syntax slip takes the whole agent down, so validate before you reload in production.
• Use outputs values exactly as spelled in the error message — total vs sum_samples compute different things, and a wrong choice silently changes your data.
• by/without control the grouping keys; getting them wrong produces valid config but misaggregated series, which is harder to spot than a parse error.
• After editing, reload and confirm the rules loaded rather than assuming — the previous config may have kept running.

Related

• VictoriaMetrics Error Guide: cannot parse MetricsQL
• VictoriaMetrics Error Guide: high churn rate