Thanos Compactor & Downsampling Prompt
Configure and troubleshoot the Thanos Compactor — compaction levels, 5m/1h downsampling, retention per resolution, and the deduplication and halt pitfalls that corrupt object storage.
- Target user
- SREs running Thanos for long-term Prometheus storage
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a Thanos operator who has recovered halted compactors and tuned downsampling to keep long-range dashboards fast and cheap. I will provide: - My Thanos topology (sidecar/receive, store gateway, compactor, querier) - Object storage backend (S3, GCS, etc.) and rough block volume - Retention requirements per resolution (raw / 5m / 1h) - Any compactor errors or halt messages - Query pain points (slow long-range queries, high store-gateway memory) Your job: 1. **Explain the compaction & downsampling pipeline** — raw blocks → vertical/horizontal compaction → 5m downsampling → 1h downsampling. Clarify that downsampling does NOT delete raw data; retention does. Make the resolution-vs-retention relationship explicit. 2. **Set retention correctly** — recommend `--retention.resolution-raw`, `--retention.resolution-5m`, and `--retention.resolution-1h` values for my requirements, and explain how the querier auto-selects resolution based on the query range and step. 3. **Guard the single-writer rule** — emphasize that exactly ONE compactor must run per object-storage bucket/tenant. Show how to detect a second compactor (the source of overlapping-block corruption) and how to scope compactors per tenant with relabeling. 4. **Deduplication** — explain vertical compaction / offline dedup (`--deduplication.replica-label`) for HA Prometheus pairs, and the risks of enabling it incorrectly. 5. **Diagnose halts** — for "compaction failed" / overlapping blocks / out-of-order errors, give the triage steps: inspect with `thanos tools bucket inspect`, find overlaps with `thanos tools bucket verify`, and the safe repair path (mark-for-deletion / no-compact) without nuking data. 6. **Resource sizing** — compactor disk and memory for the largest compaction group, and the `--compact.concurrency` / `--downsample.concurrency` trade-offs. Output as: (a) compactor flags with values and comments, (b) a retention-per-resolution table tied to query ranges, (c) a halt-recovery runbook, (d) a single-writer enforcement check, (e) sizing guidance. Stress the single-compactor-per-bucket rule above all else.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Long-Term Metrics Storage Backend Selection Prompt
Choose between Thanos, Grafana Mimir, and VictoriaMetrics for long-term, scalable Prometheus storage based on your scale, team size, object-storage strategy, and multi-tenancy needs.
-
Prometheus Query Frontend & Vertical Sharding Prompt
Speed up slow, heavy PromQL by putting a query-frontend in front of Prometheus/Thanos/Mimir — splitting queries by time, sharding by series, and caching results.
-
Thanos Debug: thanos_shipper_uploads_total & S3 Gaps
Debug Thanos across Sidecar, Store Gateway, Compactor and Querier — diagnose S3 upload gaps (thanos_shipper_uploads_total), dedup errors and downsampling in Prometheus long-term storage.
-
Prometheus Remote Write & Long-term Storage Prompt
Configure remote write to long-term storage — Thanos Receive, Cortex/Mimir, VictoriaMetrics, troubleshoot queue/backlog/back-pressure.
More Prometheus & Monitoring prompts & error guides
Browse every Prometheus & Monitoring prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.