Slack Postmortem Effectiveness Tracking Prompt
Track postmortem effectiveness in Slack — action item completion rate per postmortem, recurrence detection of same root cause, nudges for stale AIs, and quarterly trend analysis.
- Target user
- Engineering leaders measuring whether incident learning actually drives change
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior engineering leader who has built postmortem-effectiveness tracking that turned "we'll add it to the backlog" action items into measurable closure rates above 80%.
I will provide:
- Postmortem template / format
- Where postmortems live (Notion / Confluence / SharePoint)
- Where action items track (Jira / Linear / GitHub / Planner)
- Incident management tool
- Pain points (action items lost, same incidents recur, no follow-through)
Your job:
1. **What "effective" looks like** — measurable:
- Action items completed: target 80%+ within their committed window
- Recurrence rate: < 10% of incidents trace to same root-cause as a prior closed postmortem
- Time-to-postmortem-published: < 7 business days for SEV1, < 14 for SEV2
- Action item quality: each AI has owner + due + acceptance criteria
2. **Tracking spine**:
- **Postmortem registry** — every incident → postmortem (or explicit "no PM needed" decision with reason)
- **AI registry** — every AI linked to source PM + owner + due + status
- **Recurrence detection** — root cause categorization for cross-PM analysis
3. **AI capture during PM**:
- Bot scrapes postmortem document for action items
- For each: owner mention (`@user`), due date (`due:2026-07-01`), priority hint
- Bot creates tickets in tracker (Jira / Linear)
- Bot replies with confirmation: "5 AIs created from postmortem X"
4. **AI nudges**:
- 7d before due: DM owner reminder
- On due date: DM owner; status update requested
- 3d overdue: post to team channel
- 14d overdue: escalate to manager + ask for re-commit or re-prioritize
- 30d overdue: senior leader visibility + decide closeout-with-reason
5. **Closeout-with-reason flow**:
- Some AIs become "no longer applicable" or "deprioritized in favor of X"
- Require reason for non-completion
- Tracked separately from completed
- Recurring categories of reasons surfaced (e.g. "deprioritized" might signal capacity issue)
6. **Recurrence detection**:
- Each postmortem tagged with root-cause categories (DNS, secrets, config drift, dep regression, etc.)
- When new incident's RC matches prior closed PM's RC → flag in real-time:
- "This incident's RC ('expired secret') matches PM-2026-04-12; was that PM's AI completed?"
- Surfaces accountability gap
7. **Quarterly trend**:
- Total PMs by severity
- Median AI completion rate
- Categories of recurring root causes
- Teams with strongest / weakest follow-through (coaching opportunity, not blame)
- Time-to-publish trend
- Trend slope of recurrence rate (going down = learning; flat = problem)
8. **Slack visibility**:
- `#postmortem-tracking` channel for AI lifecycle events (new PM, AI due soon, AI closed, AI re-categorized)
- Weekly digest to engineering leadership
- Monthly retrospective to all engineering
9. **Dashboards**:
- **By team** — open AIs, oldest, completion rate
- **By category** — recurring root causes, AIs targeting that category
- **By severity** — SEV1 PMs vs SEV2 vs SEV3 (different patterns expected)
10. **Anti-patterns to avoid**:
- Tracking AIs without closure flow (graveyard of open tickets)
- Punishing teams for low completion (drives gaming)
- Closing AIs without verifying they actually shipped
- Ignoring closeout reasons (rich signal source)
- Postmortems published but never read
11. **Cultural overlay**:
- Reward closure, not just writing
- Senior leaders engage with the tracking signals
- Make postmortem reading part of on-call shadow
- Discuss patterns in engineering all-hands
Output as: (a) effectiveness metric definitions, (b) AI capture flow, (c) nudge timing policy, (d) closeout-with-reason workflow, (e) recurrence detection logic, (f) quarterly report schema, (g) dashboard layout, (h) cultural practices.
Bias toward: AIs as work tracked to closure, recurrence as the strongest signal of learning failure, leadership-visible metrics, blameless culture.