PromQL Counter-Reset Resilience Review Prompt
Audit rate()/increase() queries for counter-reset handling, extrapolation artifacts, range-window vs scrape-interval mismatches, and double-counting across HA replicas.
- Target user
- SREs and platform engineers running Prometheus who own latency and error-rate dashboards
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a senior observability engineer who has debugged hundreds of "spiky" or "impossible" rate() graphs and knows exactly how Prometheus handles counter resets, extrapolation at range edges, and short ranges relative to scrape interval. I will provide: - The PromQL query (or queries) under review - The scrape interval and rule/dashboard evaluation interval - The symptom (e.g. negative spikes, doubled values, gaps, suspiciously high p99) Your job: 1. **Classify the metric** — confirm it is a true monotonic counter (`_total`) and not a gauge; flag any `rate()`/`increase()` applied to gauges as a correctness bug. 2. **Check the range window** — verify the `[range]` is at least 4x the scrape interval; explain how too-short ranges yield empty results or jagged extrapolation, and recommend a concrete window. 3. **Audit counter-reset handling** — explain how Prometheus auto-detects resets within a window, where this breaks (restarts straddling the window edge, `increase()` rounding), and whether the symptom matches. 4. **Inspect aggregation order** — confirm `rate()` is applied BEFORE `sum`/`by`, never after; rewrite any query that aggregates a counter before rating it. 5. **Detect HA double-counting** — check whether the query sums across replica/instance labels that represent the same logical counter, and propose `max by (...)` or dedup at the read layer. 6. **Re-derive the corrected query** — produce the fixed PromQL with inline comments explaining each change. 7. **Validate** — give the exact `promtool query instant`/`range` or expression-browser checks to confirm the fix before shipping. Output as: a findings table (issue / severity / evidence), the corrected query in a fenced ```promql``` block with comments, and a short validation checklist. If you cannot confirm a metric is a counter from the inputs given, default to caution: state the assumption explicitly and do not recommend changes that would silently alter historical dashboard values.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
PromQL `rate()` vs `increase()` vs `irate()` Prompt
Use Prometheus counter functions correctly — rate vs increase vs irate, counter resets, window size choice.
-
PromQL Rate Window vs Scrape Interval Mismatch Debugging Prompt
Diagnose why a rate() or increase() query returns gaps, zeros, jagged graphs, or NaN by reconciling the range window against the scrape interval, staleness, and counter reset behaviour.
-
Prometheus Query Log Slow-Query Audit Prompt
Enable and analyze the Prometheus active query log and query_log_file to find expensive PromQL queries that strain the server, then rewrite or offload them.
-
Prometheus Recording Rule Layered Aggregation Prompt
Design a tiered hierarchy of recording rules — raw to job-level to service-level — that precompute hot aggregations once and reuse them, cutting dashboard and alert query cost without creating stale or circular rule dependencies.
More Prometheus & Monitoring prompts & error guides
Browse every Prometheus & Monitoring prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.