🚨 Loki Error Guide: 'Maximum active stream limit exceeded' — Cut Label Cardinality and Raise Stream Limits

Overview

Loki’s ingester rejects new streams with this error once a tenant has more active streams than its configured limit. Clients see it as an HTTP 429 from the push path:

Ingester.Push: rpc error: code = Code(429) desc = Maximum active stream limit exceeded, reduce the number of active streams (reduce labels or reduce label values), or contact your Loki administrator to see if the limit can be increased, user: 'fake'

An active stream is one unique label-value combination currently being written. Loki bounds active streams per tenant with max_global_streams_per_user (spread across ingesters) and, historically, max_streams_per_user (per ingester). This limit is the front-line defense against cardinality explosion. Hitting it almost always means a label with unbounded values — a request ID, a pod IP, a user ID — has multiplied streams far beyond what the index and ingesters can hold.

Symptoms

• New log streams stop appearing while existing ones keep flowing.
• Push clients log 429 Maximum active stream limit exceeded.
• Ingester memory and loki_ingester_memory_streams climb steeply then plateau at the cap.
• loki_discarded_samples_total with reason stream_limit increases.
• The problem appears right after a deploy that added a new label.

Common Root Causes

• High-cardinality labels — labels whose values are effectively unbounded (IDs, IPs, timestamps, full URLs).
• Dynamic label values from log content promoted to stream labels via pipeline stages.
• max_global_streams_per_user too low for a legitimately large service.
• Too few ingesters so the global limit divides into a small per-ingester share.
• A misconfigured relabel rule turning a variable field into a label.

Diagnostic Workflow

Confirm the discard reason and current stream count on an ingester:

curl -s http://loki-ingester:3100/metrics | grep -E 'loki_ingester_memory_streams|stream_limit'

Find which label is exploding cardinality using the label API or a metric query:

# List label values; a huge count reveals the culprit
logcli labels
logcli labels pod        # e.g. thousands of values = unbounded

Read the effective stream limits:

limits_config:
  max_global_streams_per_user: 5000   # total active streams per tenant, cluster-wide
  max_streams_per_user: 0             # 0 = use global only (per-ingester legacy cap)
  max_label_names_per_series: 15
  max_label_value_length: 2048

Check whether a pipeline stage is promoting a variable field to a label:

# promtail/alloy: a BAD label — request_id is unbounded
pipeline_stages:
  - labels:
      request_id:   # remove this; keep it as a log field, not a label

Example Root Cause Analysis

After a release, a service started attaching request_id as a Loki label via a Promtail labels stage. Each request produced a unique value, so within an hour the tenant accumulated tens of thousands of active streams and slammed into max_global_streams_per_user: 5000. New requests could no longer create streams and their logs were discarded with Maximum active stream limit exceeded.

The correct fix was to stop labeling by request_id — it belongs in the log line, searchable with | json | request_id="...", not as a stream label. Removing that one labels stage collapsed active streams from ~40,000 back to ~800. As genuine growth headroom they raised max_global_streams_per_user to 10000 and added a fourth ingester so the global limit spread further. Streams stabilized and discards stopped. The lesson: raising the limit would only have delayed collapse; the real bug was cardinality.

Prevention Best Practices

• Keep labels bounded and low-cardinality: use them for app, namespace, env, pod — never IDs, IPs, or URLs.
• Search variable fields with LogQL filter/parser expressions instead of promoting them to labels.
• Cap max_label_names_per_series and max_label_value_length to catch accidental explosions early.
• Set max_global_streams_per_user to real need plus headroom, and scale ingesters so the per-ingester share stays healthy.
• Review every new relabel/label pipeline stage for cardinality impact before rollout.
• Alert on loki_ingester_memory_streams trending toward the limit.

Quick Command Reference

# Active streams and stream-limit discards
curl -s localhost:3100/metrics | grep -E 'memory_streams|stream_limit'

# Spot the high-cardinality label
logcli labels
logcli labels <suspect_label> | wc -l

# Confirm running limit
curl -s http://loki:3100/config | grep max_global_streams_per_user

limits_config:
  max_global_streams_per_user: 10000
  max_label_names_per_series: 15

Conclusion

Maximum active stream limit exceeded is Loki protecting itself from cardinality explosion, and it is almost never solved by simply raising the number. Find the unbounded label — usually an ID or IP promoted to a stream label — and demote it to a searchable log field. Then set max_global_streams_per_user to real demand with headroom, scale ingesters, and alert on active stream count so the next label mistake is caught before it fills the cluster.