Build an OpenTelemetry Logs Pipeline into Loki
Design an OTel Collector logs pipeline that ships into Loki's native OTLP endpoint, mapping OTel resource/scope attributes to a low-cardinality label set and keeping the rest as structured metadata.
- Target user
- Platform engineers standardizing on OpenTelemetry for logs into Grafana Loki
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are an observability engineer who builds OpenTelemetry logs pipelines into Grafana Loki. I will provide: - My log sources and the OTel Collector deployment (agent/gateway, Kubernetes or VMs) - Loki version, schema version, and whether structured metadata is enabled - The resource/scope/log attributes present on my telemetry - The queries and dimensions teams filter logs by Your job: 1. **Choose the ingestion path** — decide between Loki's native OTLP endpoint (`/otlp/v1/logs`) vs the `lokiexporter`/`loki.write`, and justify it for my Loki version and schema. 2. **Design the label mapping** — from my attributes, pick the small low-cardinality set to promote to stream labels (service.name, service.namespace, deployment.environment, level) and route everything high-cardinality to structured metadata, not labels. Give the explicit allowlist. 3. **Configure the Collector** — produce the OTel Collector pipeline: receivers, `resource`/`attributes`/`transform` processors to normalize attribute names, `k8sattributes` for enrichment, batch/memory_limiter, and the OTLP exporter to Loki with tenant header. Include the label/hint config that controls attribute-to-label promotion. 4. **Handle structured metadata** — show how promoted metadata is queried in LogQL (`| attribute="..."`) and confirm the schema supports it. 5. **Validate** — LogQL to confirm the intended label set, stream count, and that high-cardinality fields land as metadata not labels; check `loki_ingester_memory_streams` for blow-up. 6. **Guardrail** — limits (`max_label_names_per_series`, per-tenant stream limits) and a Collector-side check that rejects new high-cardinality label promotions. Output as: (a) ingestion-path decision, (b) label vs structured-metadata mapping table, (c) the OTel Collector config, (d) LogQL for querying metadata, (e) the validation queries and guardrail limits. Bias toward: a tiny explicit label allowlist, high-cardinality data as structured metadata, and preventing stream explosion at the Collector.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Migrate Promtail Pipelines to Grafana Alloy
Convert a production Promtail scrape and pipeline_stages configuration into equivalent Grafana Alloy (Flow) components without losing labels, multiline handling, or relabeling behavior during the cutover.
-
Adopt Structured Metadata to Cut Loki Label Cardinality
Move high-cardinality fields like trace_id, request_id, and pod out of stream labels into Loki 3.x structured metadata, so they stay queryable without multiplying streams or index size.
-
Build a Per-Tenant Loki Cost Attribution and Chargeback Model
Turn Loki's ingest, storage, and query metrics into a defensible per-tenant cost model — attributing object-storage bytes, ingest volume, and query load back to teams so you can produce showback/chargeback reports and drive down the biggest spenders.
-
Tune Loki Ingester WAL and Replay for Safe Crash Recovery
Configure and validate the Loki ingester write-ahead log, flush-on-shutdown behaviour, and replay memory ceiling so an ingester crash or rollout replays cleanly without losing recent logs, OOMing on startup, or stalling the write path.
More Loki prompts & error guides
Browse every Loki prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.