Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for Prometheus & Monitoring Difficulty: Advanced ClaudeChatGPTCursor

Prometheus Native OTLP Receiver Ingestion Prompt

Enable and tune Prometheus's built-in OTLP metrics receiver (/api/v1/otlp/v1/metrics) so OpenTelemetry SDKs and Collectors can push straight into Prometheus with correct name translation, resource-attribute promotion, and out-of-order handling — without a separate remote_write bridge.

Target user
Observability engineer consolidating OpenTelemetry metrics into Prometheus/TSDB as the push-based ingestion endpoint
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a senior observability engineer who has migrated push-based OpenTelemetry pipelines onto Prometheus's native OTLP receiver and knows the naming, staleness, and out-of-order trade-offs that come with a push model on a pull-first TSDB.

I will provide:
- Prometheus version and how it's launched (flags, config file)
- Who is pushing OTLP (OTel SDK direct, or an OpenTelemetry Collector otlphttp exporter) and over what protocol (HTTP/protobuf)
- Which resource attributes I care about keeping as labels (service.name, service.namespace, k8s.*, deployment.environment)
- Symptoms if any (metrics missing, weird `_total` suffixes, out-of-order rejects, unexpected `target_info` series)

Your job:

1. **Turn the receiver on correctly.** Show that the OTLP receiver is gated behind the feature flag / `otlp` config section for the given version, that it requires the write path to be enabled, and give the exact endpoint clients push to (`/api/v1/otlp/v1/metrics`). Note it accepts HTTP protobuf, not the Prometheus remote_write protobuf.

2. **Get naming right.** Explain the `translation_strategy` options (e.g. `UnderscoreEscapingWithSuffixes` vs `NoUTF8EscapingWithSuffixes`) — what unit/`_total` suffixing and dot-to-underscore escaping each does, and when to prefer UTF-8 names. Show what a metric like `http.server.request.duration` becomes under each.

3. **Promote resource attributes.** Configure `promote_resource_attributes` so the resource attributes I listed become labels on every series, and explain that unpromoted attributes land on a `target_info` metric that must be joined via `group_left` — with an example PromQL join.

4. **Handle out-of-order and staleness.** Push clients don't get Prometheus's staleness markers, so explain enabling `out_of_order_time_window` on the TSDB for late OTLP batches, the cardinality/memory cost, and why absent()/alerting behaves differently for pushed series than scraped ones.

5. **Decide receiver vs Collector remote_write.** Give a short decision guide: when to push OTLP straight into Prometheus vs. keep an OTel Collector doing `prometheusremotewrite`, considering batching, retries, backpressure, and multi-tenancy.

Output as: (a) the Prometheus flags + `otlp` config block with comments, (b) the matching OTel Collector `otlphttp` exporter snippet, (c) one PromQL example joining `target_info` for an unpromoted attribute, (d) a 3-line rollout note.

Do not enable a wide `out_of_order_time_window` on a busy server without estimating the head-series and memory impact first, and validate name translation in a staging TSDB before repointing production traffic.

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Related prompts

More Prometheus & Monitoring prompts & error guides

Browse every Prometheus & Monitoring prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.