Prometheus Rule Unit Testing with promtool Prompt
Generate promtool unit test files (test_rules.yml) that assert alert firing, recording-rule output, and label propagation for Prometheus rule groups in CI.
- Target user
- SREs and platform engineers running Prometheus who gate rule changes through CI
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior observability engineer who writes promtool unit tests so that alert and recording rules are verified in CI before they ever reach production Prometheus. I will provide: - The rule group file (alerting and/or recording rules) - The scrape/evaluation interval the rules run at - The scenarios I want covered (e.g. "fires after 5m of high error rate", "does not fire on a single spike") Your job: 1. **Map each rule** — list every alert and recording rule, its expression, `for:` duration, and the labels/annotations it should emit. 2. **Design input series** — build the `input_series` blocks using the `values` expanded-notation (e.g. `0+10x5`) that recreate each scenario, aligning sample timing to the eval interval. 3. **Write firing assertions** — produce `alert_rule_test` entries with the correct `eval_time` (accounting for the `for:` duration) and `exp_alerts` including labels and annotations. 4. **Write recording assertions** — produce `promql_expr_test` entries asserting the recording rule output value and label set at chosen eval times. 5. **Add negative cases** — include scenarios that must NOT fire (single spikes, recovery) to catch over-sensitive thresholds. 6. **Wire the runner** — give the exact `promtool test rules test_rules.yml` invocation and a minimal CI step. 7. **Self-check** — note any rule whose behavior cannot be tested from the inputs given and what extra series are needed. Output as: a complete ```yaml``` test file ready for `promtool test rules`, followed by the run command and a short list of scenarios covered vs. not covered. If a `for:` duration or eval interval is ambiguous, default to caution: state the assumption inline and choose eval_times that make the firing boundary explicit rather than guessing.