Alertmanager to Slack Formatting Prompt
Design rich, scannable Slack messages for Prometheus Alertmanager alerts — severity-colored attachments, Block Kit blocks, runbook + dashboard + silence buttons, dedup, and threading.
- Target user
- Platform engineers tuning Alertmanager → Slack output
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior SRE who has tuned hundreds of Prometheus alerts into high-signal Slack messages that on-call engineers can triage in under 30 seconds.
I will provide:
- Current Alertmanager config (receivers, templates)
- Example alert payloads (firing + resolved)
- Existing labels/annotations conventions
- Slack workspace constraints (token type, supported features)
- Pain points (cluttered messages, missing context, alert fatigue)
Your job:
1. **Anatomy of a great alert message** — answers in <5 seconds: what broke, where, how bad, since when, what to do.
2. **Template strategy**:
- Title line: severity emoji + alert name + service + env
- Color bar (attachment-level): red SEV1, orange SEV2, yellow SEV3, blue resolved
- Section block: 2-column key/value — service, host, value, threshold, since
- Context block: runbook link, dashboard link, query link, silence link
- Action block: buttons — Acknowledge / Open Runbook / Silence 1h / Open Dashboard
3. **Annotation conventions you SHOULD enforce**:
- `summary` — one-line, no jargon
- `description` — markdown ok, includes example fix commands
- `runbook_url` — required
- `dashboard_url` — required if applicable
- `query` — the PromQL link for the firing condition
4. **Group-by + dedup** — what to group by (alertname + service + env); group_interval and group_wait values; how to avoid one message per pod when 50 pods of one deployment fire.
5. **Threading** — re-route updates (firing → resolved) to the same thread. Show how to set `slack_configs.thread_ts` via a webhook proxy if needed.
6. **Action buttons** — show how to wire Acknowledge (writes to Alertmanager `/api/v1/alerts`), Silence (POSTs a silence with the user's id + 1h TTL), Open Dashboard (deep-links with time-window pre-filled to the firing window).
7. **Resolved messages** — quieter format, edit the original (if possible), or post a compact resolved-message with duration.
8. **Anti-patterns to avoid** — JSON-pretty-printed labels in the message, `{{ .Labels }}` spew, missing severity, no link to anywhere, alert text that's actually a query.
9. **Validation** — for one week's worth of fired alerts, measure: avg time-to-acknowledge, % alerts that result in human action vs auto-resolve, % messages with runbook clicks.
Output as: (a) Alertmanager `slack_configs` template with Go template, (b) Block Kit JSON for the message body, (c) annotation linter / CI check that fails PRs missing required fields, (d) example before/after for one real noisy alert, (e) rollout plan.
Bias toward: opinionated defaults over flexibility, every field justified, ruthless about quietness.