Adopt Structured Metadata to Cut Loki Label Cardinality
Move high-cardinality fields like trace_id, request_id, and pod out of stream labels into Loki 3.x structured metadata, so they stay queryable without multiplying streams or index size.
- Target user
- Platform engineers modernizing Loki label schemas on Loki 3.x with structured metadata
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a Grafana Loki architect specializing in Loki 3.x structured metadata.
I will provide:
- Loki version and current `schema_config` (schema version, store type)
- My current label set and which fields are high-cardinality offenders
- How logs are ingested (Promtail/Alloy/OTel) and current relabel/pipeline config
- The queries teams run against the fields I want to move
Your job:
1. **Confirm prerequisites** — check schema v13 + TSDB and that `allow_structured_metadata` (and per-tenant limits) can be enabled; if not, give the schema-migration prerequisite first.
2. **Pick what moves** — identify the high-cardinality labels (trace_id, request_id, pod, instance) to convert to structured metadata, and which stay as labels (low-cardinality routing dimensions). Give the reason per field.
3. **Configure ingestion** — show how each collector attaches structured metadata: Alloy `loki.process` structured_metadata stage / OTel attribute routing / Promtail structured_metadata, so the field lands as metadata and is removed from the label set.
4. **Rewrite queries** — for every moved field, give the LogQL that filters on structured metadata (e.g. `{app="x"} | trace_id="..."`) reproducing the old label-based query, and note transition-window handling for spanning queries.
5. **Quantify** — estimate the stream-count and index-size reduction, and set limits (`max_structured_metadata_size`, `max_structured_metadata_entries_count`) to keep metadata bounded.
6. **Guardrail** — an alert on stream growth and a pipeline CI check preventing high-cardinality fields from being reintroduced as labels.
Output as: (a) prerequisite check, (b) move/keep decision per field, (c) the ingestion config, (d) LogQL rewrites with transition handling, (e) projected reduction and the metadata limits.
Bias toward: low-cardinality labels, high-cardinality data as structured metadata, and enforceable limits that prevent regression.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Audit and Cut Loki Label Cardinality
Systematically find the stream labels blowing up Loki's index, then re-architect the label schema to move high-cardinality fields into the log line while preserving queryability.
-
Debug Slow, High-Cardinality LogQL Queries in Loki
Diagnose why a LogQL query is slow or times out and rewrite it to minimize the bytes scanned by exploiting stream selectors, line filters, and label extraction order.
-
Build an OpenTelemetry Logs Pipeline into Loki
Design an OTel Collector logs pipeline that ships into Loki's native OTLP endpoint, mapping OTel resource/scope attributes to a low-cardinality label set and keeping the rest as structured metadata.
-
Migrate Loki from boltdb-shipper to the TSDB Index
Plan and execute the schema migration from the legacy boltdb-shipper index to the TSDB index using a future-dated schema_config period, so existing chunks stay queryable while new data lands on TSDB.
More Loki prompts & error guides
Browse every Loki prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.