Prometheus Experimental Feature-Flag Rollout Prompt
Plan a safe rollout of an experimental Prometheus feature enabled via --enable-feature, assessing risk, dependencies, and rollback before turning it on in production.
- Target user
- Platform lead deciding whether and how to enable an experimental Prometheus feature flag in production
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior observability engineer who has shepherded experimental Prometheus features into production without paging the on-call. I will provide: - The feature(s) I want to enable (e.g. exemplar-storage, memory-snapshot-on-shutdown, promql-experimental-functions, auto-gomemlimit, otlp-deltatocumulative, native-histograms behavior) - My Prometheus version - My environment topology (HA pair, agent + remote_write, single server, sharded) - My change-management constraints and rollback expectations Your job: 1. **Pin to version** — confirm whether the requested `--enable-feature` flag exists, is still experimental, has been promoted to stable (no longer needing the flag), or removed in my Prometheus version, and note exactly how the flag is spelled. 2. **State the blast radius** — explain what the feature changes operationally (storage format, query behavior, startup time, memory, on-disk compatibility) and whether enabling it writes data that is incompatible with rolling back. 3. **Identify dependencies** — call out anything the feature requires (e.g. exemplar-storage needs OpenMetrics scraping; certain features interact with remote_write or native histograms) so it is not enabled in isolation and broken. 4. **Design the rollout** — give a staged plan: enable on a canary replica or a non-critical shard first, define success/failure signals, and a defined soak period before fleet-wide. 5. **Define rollback** — specify the exact rollback (remove flag + restart) and whether any persisted data must be cleaned, plus a verification that the server returns to healthy. Output as: (a) a feature status line (experimental/stable/removed for my version), (b) a risk table (what changes, rollback-safe?, dependencies), (c) the staged rollout plan with go/no-go signals, (d) the rollback procedure. Do not enable experimental flags on both HA replicas simultaneously — keep one on the known-good config until the canary soaks.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Prometheus keep_dropped_targets Limit Tuning Prompt
Set keep_dropped_targets on service-discovery-heavy Prometheus servers to cap the memory spent retaining metadata for relabel-dropped targets, so a churny SD source (Kubernetes, EC2, Consul) can't quietly balloon scrape-manager memory or the /api/v1/targets response.
-
Prometheus OAuth2 Scrape Target Authorization Prompt
Configure and troubleshoot OAuth2 client-credentials authorization on Prometheus scrape jobs so that Prometheus can pull /metrics from targets sitting behind an OIDC-protected gateway without leaking secrets or hammering the token endpoint.
-
Prometheus Native OTLP Receiver Ingestion Prompt
Enable and tune Prometheus's built-in OTLP metrics receiver (/api/v1/otlp/v1/metrics) so OpenTelemetry SDKs and Collectors can push straight into Prometheus with correct name translation, resource-attribute promotion, and out-of-order handling — without a separate remote_write bridge.
-
Prometheus honor_labels & honor_timestamps Conflict Resolution Prompt
Diagnose and fix label collisions and timestamp drift caused by honor_labels/honor_timestamps when scraping federation endpoints, Pushgateway, or exporters that expose their own job/instance labels.
More Prometheus & Monitoring prompts & error guides
Browse every Prometheus & Monitoring prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.