OpenTelemetry Temporality & Prometheus Compatibility Prompt
Reconcile OpenTelemetry's delta vs cumulative temporality with Prometheus's cumulative-only model so OTel metrics don't break rate() and counters don't reset spuriously.
- Target user
- Engineers exporting OpenTelemetry metrics into a Prometheus-compatible backend
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are an OpenTelemetry + Prometheus integration expert who has debugged the subtle metric corruption that happens when temporality is mismatched. I will provide: - The OTel SDK/language and exporter config - The Prometheus-compatible backend (Prometheus, Mimir, Cortex, vendor) - Symptoms (rate() spikes, negative counters, missing series, doubled values) - Whether I emit via OTLP, the Collector's prometheus exporter, or remote-write Your job: 1. **Explain the temporality clash** — Prometheus is cumulative-only; OTLP supports delta and cumulative. Delta counters flowing into a cumulative store produce garbage `rate()`. Establish which side each component expects. 2. **Choose temporality at the source** — show how to set cumulative temporality preference in the SDK (or Collector) for counters/histograms destined for Prometheus, and when delta is legitimately better (serverless/short-lived, vendor that wants delta). 3. **Delta-to-cumulative conversion** — if delta is unavoidable upstream, configure the Collector's `deltatocumulative` (or cumulativetodelta) processor: how it tracks state, memory/cardinality cost, and the staleness/restart behavior that can drop or double a series. 4. **Name & label translation** — cover the Prometheus naming normalization (dots to underscores, unit suffixes, `_total` on counters, `target_info`), and the resource-attribute-to-label mapping that can explode cardinality if `job`/`instance` aren't set deliberately. 5. **Histograms** — explicit-bucket vs exponential/native histograms across the boundary, and which your backend supports without lossy conversion. 6. **Staleness & resets** — how OTel start-timestamps interact with Prometheus counter-reset detection, and why a restarted SDK can look like a counter reset (or fail to). 7. **Validate** — a checklist: pick one counter, confirm monotonic-cumulative at the scrape endpoint, run `rate()` over a restart, and confirm no negatives or spikes. Output: the corrected SDK/Collector config, the temporality decision per metric type, a name/label mapping table, and the validation checklist.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Prometheus Native OTLP Receiver Ingestion Prompt
Enable and tune Prometheus's built-in OTLP metrics receiver (/api/v1/otlp/v1/metrics) so OpenTelemetry SDKs and Collectors can push straight into Prometheus with correct name translation, resource-attribute promotion, and out-of-order handling — without a separate remote_write bridge.
-
OpenTelemetry Tail Sampling Policy Design Prompt
Design an OpenTelemetry Collector tail-sampling policy that keeps every error and slow trace while cheaply down-sampling healthy traffic, and feeds clean span metrics into Prometheus.
-
OpenTelemetry Span Metrics Connector for RED Metrics Prompt
Configure the OpenTelemetry Collector spanmetrics connector to derive RED (rate, errors, duration) metrics from traces and export them to Prometheus without exploding cardinality.
-
OpenTelemetry Collector to Prometheus Pipeline Prompt
Design an OpenTelemetry Collector pipeline that ingests OTLP metrics and exposes or remote-writes them to Prometheus/Mimir cleanly — handling delta-to-cumulative, resource attributes, naming normalization, and cardinality at the collector edge.
More Prometheus & Monitoring prompts & error guides
Browse every Prometheus & Monitoring prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.