Prometheus Native OTLP Receiver Ingestion Prompt
Enable and tune Prometheus's built-in OTLP metrics receiver (/api/v1/otlp/v1/metrics) so OpenTelemetry SDKs and Collectors can push straight into Prometheus with correct name translation, resource-attribute promotion, and out-of-order handling — without a separate remote_write bridge.
- Target user
- Observability engineer consolidating OpenTelemetry metrics into Prometheus/TSDB as the push-based ingestion endpoint
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior observability engineer who has migrated push-based OpenTelemetry pipelines onto Prometheus's native OTLP receiver and knows the naming, staleness, and out-of-order trade-offs that come with a push model on a pull-first TSDB. I will provide: - Prometheus version and how it's launched (flags, config file) - Who is pushing OTLP (OTel SDK direct, or an OpenTelemetry Collector otlphttp exporter) and over what protocol (HTTP/protobuf) - Which resource attributes I care about keeping as labels (service.name, service.namespace, k8s.*, deployment.environment) - Symptoms if any (metrics missing, weird `_total` suffixes, out-of-order rejects, unexpected `target_info` series) Your job: 1. **Turn the receiver on correctly.** Show that the OTLP receiver is gated behind the feature flag / `otlp` config section for the given version, that it requires the write path to be enabled, and give the exact endpoint clients push to (`/api/v1/otlp/v1/metrics`). Note it accepts HTTP protobuf, not the Prometheus remote_write protobuf. 2. **Get naming right.** Explain the `translation_strategy` options (e.g. `UnderscoreEscapingWithSuffixes` vs `NoUTF8EscapingWithSuffixes`) — what unit/`_total` suffixing and dot-to-underscore escaping each does, and when to prefer UTF-8 names. Show what a metric like `http.server.request.duration` becomes under each. 3. **Promote resource attributes.** Configure `promote_resource_attributes` so the resource attributes I listed become labels on every series, and explain that unpromoted attributes land on a `target_info` metric that must be joined via `group_left` — with an example PromQL join. 4. **Handle out-of-order and staleness.** Push clients don't get Prometheus's staleness markers, so explain enabling `out_of_order_time_window` on the TSDB for late OTLP batches, the cardinality/memory cost, and why absent()/alerting behaves differently for pushed series than scraped ones. 5. **Decide receiver vs Collector remote_write.** Give a short decision guide: when to push OTLP straight into Prometheus vs. keep an OTel Collector doing `prometheusremotewrite`, considering batching, retries, backpressure, and multi-tenancy. Output as: (a) the Prometheus flags + `otlp` config block with comments, (b) the matching OTel Collector `otlphttp` exporter snippet, (c) one PromQL example joining `target_info` for an unpromoted attribute, (d) a 3-line rollout note. Do not enable a wide `out_of_order_time_window` on a busy server without estimating the head-series and memory impact first, and validate name translation in a staging TSDB before repointing production traffic.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Prometheus Out-of-Order Sample Ingestion Tuning Prompt
Configure and tune out-of-order sample ingestion (tsdb.out_of_order_time_window) to accept delayed/backfilled samples without breaking compaction or exploding memory.
-
OpenTelemetry Collector to Prometheus Pipeline Prompt
Design an OpenTelemetry Collector pipeline that ingests OTLP metrics and exposes or remote-writes them to Prometheus/Mimir cleanly — handling delta-to-cumulative, resource attributes, naming normalization, and cardinality at the collector edge.
-
Prometheus TSDB Head Memory & Series Churn Prompt
Diagnose Prometheus memory pressure driven by the in-memory head block, distinguishing high active-series load from high series churn, and applying the right remediation for each.
-
Prometheus WAL Replay Startup Latency Prompt
Diagnose and reduce slow Prometheus startup caused by long write-ahead-log (WAL) replay, so a restarting server returns to a healthy, scrapeable state quickly after deploys or crashes.
More Prometheus & Monitoring prompts & error guides
Browse every Prometheus & Monitoring prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.