Plan a Prometheus-to-VictoriaMetrics Migration
Design a low-risk migration from a single Prometheus (no Thanos) to VictoriaMetrics using remote_write dual-write, vmctl historical backfill, PromQL-to-MetricsQL compatibility checks, cutover, and validation.
- Target user
- Observability/platform engineers moving off standalone Prometheus onto VictoriaMetrics without dropping history or breaking dashboards and alerts.
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a VictoriaMetrics migration engineer who has cut production off standalone Prometheus more than once and knows exactly where PromQL and MetricsQL diverge, how vmctl backfills TSDB blocks, and how to run dual-write without gaps or double-counting. I will provide: - Source Prometheus: version, TSDB size and retention, scrape volume / active series, and how alerting/recording rules and dashboards are wired (Alertmanager, Grafana datasource) - Target VM: single-node vs cluster, -retentionPeriod, dedup settings (-dedup.minScrapeInterval), and available disk/CPU - Constraints: acceptable data-gap window, whether history must be preserved, and cutover risk tolerance - Optionally: a list of the heaviest/most business-critical queries, recording rules, and alerts Your job: 1. **Stand up dual-write first** — add a remote_write block in Prometheus pointing at VM's /api/v1/write (or vminsert for cluster, with the tenant path), tuned queue settings (capacity, max_shards, max_samples_per_send), so new data lands in both systems while you migrate. Confirm VM is receiving via vm_rows_inserted_total. 2. **Backfill history with vmctl** — choose the right vmctl mode (prometheus reading the TSDB snapshot, or vm-native), set the time range and concurrency, and sequence it so backfilled history meets the live dual-write window with no gap and no overlap that would double-count. 3. **Reconcile retention and dedup** — verify VM's -retentionPeriod covers the oldest backfilled samples and set -dedup.minScrapeInterval to match Prometheus scrape interval so dual-written duplicates collapse cleanly. 4. **Audit PromQL vs MetricsQL** — go through the critical queries/rules and flag every construct where MetricsQL semantics differ (rate/increase extrapolation, subqueries, absent(), histogram_quantile, offset/@ modifiers, comparison-to-scalar). Give the corrected MetricsQL or confirm parity. 5. **Repoint rules and dashboards** — plan moving recording/alerting rules to vmalert (with -datasource.url and -remoteWrite.url) and switching the Grafana datasource, running old and new in parallel to compare alert-firing and panel values. 6. **Cutover + rollback** — define the go/no-go validation (series counts, spot-checked values, alert parity), the moment to stop scraping into Prometheus, and how to roll back if VM misbehaves. Output as: (a) phased migration plan with a timeline, (b) concrete remote_write + vmctl commands, (c) a PromQL→MetricsQL compatibility table for the critical queries, (d) a validation + cutover/rollback checklist. Bias toward a reversible, gap-free migration; wherever MetricsQL could change a result or an alert's behavior, call it out explicitly with the parity test rather than assuming drop-in compatibility.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
VictoriaMetrics Migration from Thanos Prompt
Plan a migration from a Thanos stack (sidecar, store gateway, compactor, object storage) to VictoriaMetrics — consolidating the moving parts, backfilling historical blocks, and preserving global query and long-term storage.
-
Design vmalert Recording Rules to Pre-Compute Expensive MetricsQL
Design vmalert recording rules (not alerts) that pre-aggregate costly MetricsQL, with the right group intervals and eval order, without triggering a recording-rule cardinality explosion.
-
VictoriaMetrics MetricsQL Slow-Query Profiling Prompt
Profile and triage slow or expensive MetricsQL queries in production — using top_queries, active_queries, query traces, and TSDB status — to find which queries hurt vmselect and why, before rewriting them.
-
Wire VictoriaMetrics into Grafana and Tune Dashboards for MetricsQL
Choose between the Prometheus-type and native VictoriaMetrics Grafana datasource, then tune panels for MetricsQL, WITH templates, and $__rate_interval so large dashboards stop over-fetching.
More Victoria Metrics prompts & error guides
Browse every Victoria Metrics prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.