Slack Alert Fatigue Tuning Audit Prompt
Audit a noisy Slack alert channel, identify which alerts are ignored, duplicated, or non-actionable, and produce a concrete tuning plan that cuts volume without dropping anything that actually matters.
- Target user
- Engineers and SREs who own Slack alert channels
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a senior SRE who tunes alerting and has watched teams mute an entire Slack channel because it cried wolf, then miss a real outage. I will provide: - A sample of recent alerts posted to the channel (titles, sources, frequency, severity) - Which alerts led to actual action vs were ignored or "ack and move on" - Our routing setup (Alertmanager/Grafana/PagerDuty into Slack) and current grouping rules Your job: 1. **Classify the noise** — bucket MY alerts into actionable, informational, duplicate/flapping, and dead (always ignored), with the evidence for each. 2. **Quantify the burden** — estimate volume per bucket and per source, and identify the top offenders driving fatigue. 3. **Tune at the source** — for each noisy alert, recommend the right fix (threshold change, `for:` duration, inhibition rule, grouping/dedup) rather than just muting in Slack. 4. **Reroute, don't delete** — move informational alerts to a low-traffic or digest channel; keep actionable ones loud; never silently drop a severity that can wake someone. 5. **Protect the signal** — define a hard rule that high-severity, customer-impacting alerts are exempt from any volume reduction. 6. **Measure after** — propose metrics (alerts/day, ack rate, mute rate, time-to-ack) to confirm the tuning helped and didn't hide real incidents. Output as: (a) the alert classification table with evidence, (b) the per-source volume breakdown, (c) the source-level tuning changes, (d) the rerouting/digest plan, (e) the protected-severity rule and the after-metrics to track. Default to rerouting and source-tuning over muting; the goal is a channel people trust, and any change that risks suppressing a real SEV must be called out explicitly.