🚨 Loki Error Guide: 'has 25 label names; limit 15' — Cut Label Cardinality Before the Distributor Rejects Your Push

What this error means

Loki’s distributor validates every stream label set as it ingests a push. When a stream carries more labels than the configured ceiling, the distributor rejects the push with an HTTP 400 and a message like this:

stream '{app="api", ...}' has 25 label names; limit 15

The limit comes from limits_config.max_label_names_per_series, which defaults to 15. Every unique combination of label names and values is a distinct stream, so a high label count is not just a validation failure — it is a warning that you are about to explode Loki’s stream cardinality. When the distributor drops these pushes, it increments loki_discarded_samples_total{reason="max_label_names_per_series"}, which is the metric to watch and alert on.

The fix is almost never to raise the limit. It is to stop your log agent from attaching high-cardinality labels like pod, container, node, ip, and request_id, which belong in the log line or in structured metadata, not in the stream label set.

Warning signs

• Pushes fail with HTTP 400 and has N label names; limit 15 in the distributor logs or in the agent’s send errors.
• loki_discarded_samples_total{reason="max_label_names_per_series"} climbs steadily.
• A subset of applications (usually the noisiest ones) lose logs while others ingest fine.
• Promtail/Alloy client metrics show increasing dropped or failed batches for specific targets.
• Stream and active-series counts trend upward right before the rejections begin.

Measuring the limit

1. Confirm the rejection reason in the distributor logs so you are not chasing a different validation error:

   kubectl logs -l app=loki,component=distributor -n loki --tail=300 \
     | grep 'label names; limit'

2. Quantify the discards by reason to confirm this is the dominant drop cause:

   sum by (reason) (
     rate(loki_discarded_samples_total{reason="max_label_names_per_series"}[5m])
   )

3. List the offending stream’s labels to see exactly which labels are being attached and which are high-cardinality:

   logcli series '{app="api"}' --analyze-labels \
     --addr=https://loki-gateway/ --org-id=tenant-a

4. Read the effective limit so you know the real ceiling in force for the tenant:

   limits_config:
     max_label_names_per_series: 15

5. Inspect the agent pipeline that produces the stream. In Promtail/Alloy, find the stages that set labels and identify which are stable (keep) versus dynamic (drop):

   pipeline_stages:
     - json:
         expressions:
           level: level
           request_id: request_id
     - labels:
         level:
         request_id:   # high-cardinality: this is the problem

Limits and pressure causes

• Log agent attaching per-request labels — the pipeline promotes request_id, trace_id, or session_id to labels, so every request becomes its own stream and the label count balloons.
• Kubernetes metadata over-labeling — pod, container, node, namespace, ip, and image are all promoted to labels when only namespace/app are stable enough to belong there.
• Copying every JSON field into labels — a stage.json or stage.labels block that maps the entire parsed structure into labels instead of a curated subset.
• Dynamic infrastructure labels — autoscaled node names, ephemeral IPs, or container IDs that change constantly and inflate both the label count and the stream count.
• Well-intentioned “add context” changes — someone adds a couple of labels for one debugging session and never removes them, pushing a busy stream over the ceiling.
• Multi-tenant defaults left unbounded — a shared limits_config where one tenant’s agent config is far more aggressive than the platform default assumes.

Remediation

Drop high-cardinality labels in Promtail before sending — keep only stable, low-cardinality labels like namespace, app, and level, and remove per-request identifiers with labeldrop:

pipeline_stages:
  - labeldrop:
      - request_id
      - pod
      - container
      - ip
      - node

Do the same in Grafana Alloy with a relabel rule that keeps only the labels you actually query on:

loki.relabel "keep_stable" {
  forward_to = [loki.write.default.receiver]
  rule {
    action        = "labeldrop"
    regex         = "request_id|pod|container|ip|node|image_id"
  }
}

Move dynamic fields into the log line, not labels — leave request_id and friends in the JSON body and extract them at query time, so they cost nothing at ingest:

{namespace="prod", app="api"} | json | request_id="abc-123"

Use structured metadata for high-cardinality context — attach fields like trace_id as structured metadata rather than stream labels, which keeps cardinality flat while remaining queryable:

limits_config:
  allow_structured_metadata: true

Raise the limit only as a deliberate, temporary stopgap — bump it just enough to unblock ingestion while you fix the agent, and treat the raise as debt to pay down:

limits_config:
  max_label_names_per_series: 20   # temporary; reduce after agent cleanup

Capacity headroom

• Raising max_label_names_per_series does not fix cardinality — it lets more streams through, which can quietly overload ingesters and blow up your index and memory.
• request_id, trace_id, ip, and timestamps are classic anti-pattern labels; they should live in the log body or structured metadata, never in the stream label set.
• A per-tenant override can mask the problem for one tenant while the platform default still bites everyone else; keep tenant limits explicit and reviewed.
• Dropped pushes are unrecoverable — the agent may retry, but if the label set is still over the limit those retries also fail and the logs are lost.
• Watch loki_discarded_samples_total{reason="max_label_names_per_series"} continuously; a sudden rise usually means someone shipped a new labeling change.

Related capacity errors

• Loki Error Guide: ‘Maximum active stream limit exceeded’ — the stream-count blowup that too many labels directly causes.
• Loki Error Guide: ‘Per stream rate limit exceeded’ — the throughput side of the same over-labeling problem.
• Loki Error Guide: ‘maximum of series reached’ — the query-time cost of exploded cardinality.