Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for Victoria Metrics Difficulty: Advanced ClaudeChatGPTCursor

VictoriaMetrics MetricsQL Slow-Query Profiling Prompt

Profile and triage slow or expensive MetricsQL queries in production — using top_queries, active_queries, query traces, and TSDB status — to find which queries hurt vmselect and why, before rewriting them.

Target user
Observability engineers triaging vmselect latency and query load on VictoriaMetrics
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a VictoriaMetrics read-path SRE who profiles query load for a living and knows vmselect's introspection endpoints, query tracing, and TSDB cardinality tooling cold.

I will provide:
- The symptom: vmselect latency spikes, `503`/timeout errors, memory pressure, or slow dashboards
- Whatever telemetry I have: `/api/v1/status/top_queries` output, `/api/v1/status/active_queries`, query traces, slow-query log lines, or `vm_request_duration_seconds` panels
- Deployment shape: single-node vs. cluster, number of vmselect nodes, and current `-search.*` limits
- Optionally: `/api/v1/status/tsdb` cardinality output for suspect metrics

Your job:

1. **Rank the offenders** — from `top_queries` (by average duration and by frequency) and `active_queries`, identify the queries doing the most damage. Separate "one expensive query run rarely" from "a cheap-ish query run constantly," because the fix differs.

2. **Attribute the cost** — for each top offender, explain what makes it expensive: scanned series count, time range × step (points per series), high-cardinality `by`/`without` grouping, subqueries, or unbounded matchers. Tie the explanation to the trace or TSDB status where I have it.

3. **Read a query trace** — if I paste an `EXPLAIN`/trace, walk its stages (series selection, rollup, aggregation) and point to where the time and memory actually go, so I'm not optimizing the wrong step.

4. **Correlate with limits and resources** — relate the findings to `-search.maxConcurrentRequests`, `-search.maxPointsPerTimeseries`, `-search.maxSeries`, and vmselect memory, so I know whether I'm hitting a limit, saturating concurrency, or exhausting RAM.

5. **Prescribe the triage action, not just a rewrite** — for each offender recommend the right lever: move to a recording rule, tighten the matcher, cap the range/step, raise a limit, add a vmselect, or cache. Say which gives the biggest latency win for the least risk.

6. **Set up standing observability** — give me the queries/alerts to keep this visible: p99 `query_range` latency, `vm_concurrent_select_current / capacity`, and a periodic `top_queries` review, so the next regression is caught before users file a ticket.

Output as: (a) ranked offender list with cost attribution, (b) trace walkthrough (if provided), (c) limit/resource correlation, (d) per-offender triage action with expected impact, (e) standing observability queries/alerts.

Bias toward measuring before rewriting; if the data doesn't yet show which query is at fault, tell me exactly which endpoint or trace to capture next rather than guessing.

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Related prompts

More Victoria Metrics prompts & error guides

Browse every Victoria Metrics prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.