Skip to content
🎉 Launch sale:50% off everything over $22 — automatically applied at checkout· ends Aug 2Shop the sale →
DevOps AI ToolKit
Newsletter
All guides
Post Mortems with AI By James Joyner IV · · 9 min read

From Postmortem to Prevention: Closing the Loop

A postmortem only pays off if the fixes ship and actually work. Here's how to close the loop from findings to verified prevention and stop repeat incidents.

  • #postmortems
  • #sre
  • #reliability
  • #incident-response
  • #prevention

The point of a postmortem is not the postmortem. It’s the moment, months later, when a would-be incident quietly doesn’t happen because you fixed the thing that caused the last one. Everything between writing the doc and that non-event is the loop most teams fail to close. They analyze well, file good action items, and then watch the same class of incident recur because the fixes never shipped or never actually worked. This guide is about closing that loop.

The loop, end to end

Prevention is a chain, and it breaks at the weakest link:

Finding → Action item → Prioritized → Shipped → Verified → Recurrence checked

Most teams do the first two links well and lose the rest. Let’s walk each.

Covered in depth elsewhere, so briefly: each action item must be specific, owned by one person, tracked in your real backlog, dated, and classified as prevent/detect/mitigate. The critical move here is resizing. A finding like “the deploy pipeline has no validation” often implies a big project. A project stalls. Break out the specific, shippable slice this postmortem justifies — “add a lint rule that blocks locking migrations” — and file that. The rest can be a separate backlog item, but the postmortem’s contribution has to be small enough to actually land.

This is where prevention work goes to die, quietly. Postmortem action items compete with features, and features usually win because they have a louder champion. To keep reliability work alive:

  • Put action items in the same backlog as feature work, visible in the same planning. Work in a separate “reliability doc” nobody looks at during planning doesn’t exist.
  • Give high-severity findings real priority, not aspirational later dates. If the incident was a SEV-1, its prevention items earn a slot this sprint or you’re accepting the risk of a repeat — say so out loud.
  • Make the trade-off explicit. “We’re deferring the pool-sizing fix to next quarter” is a legitimate decision if someone with authority makes it knowingly. Silent deferral is how repeat incidents happen.

An action item marked “in progress” for two months isn’t prevention. Close this link with a standing review:

  • Review open postmortem action items on a schedule — a weekly reliability sync or a recurring agenda item — where someone reads out the overdue ones. Sunlight is the cheapest project management there is.
  • Escalate the stuck ones. An item overdue by weeks is mis-prioritized, blocked, or badly sized. Each of those needs a decision, not another week of silence.
  • Track completion rate as a health metric. If under half of last quarter’s items shipped, the loop is broken upstream and no amount of individual nagging fixes it.

Here’s the one that catches sophisticated teams: an action item marked “done” is not the same as a problem fixed. Someone added an alert — does it actually fire on the condition that fooled you last time? Someone added a validation step — does it actually block the bad input? “Done” often means “code merged,” not “verified to prevent recurrence.”

Close this link with verification, proportional to severity:

  • Test the fix against the original failure. If you added a timeout to survive a slow dependency, inject latency in staging and confirm the system degrades gracefully. If you added an alert, trigger the condition and confirm it pages.
  • Prefer proof over belief. A fix nobody has exercised is a hypothesis. For high-severity incidents, run a game-day or fault injection that reproduces the original conditions.
  • Note verification in the action item. “Verified 08-02: injected 5s latency in staging, circuit breaker opened as expected.” Now it’s actually closed.

The final link is longitudinal. Periodically ask: has this class of incident recurred since we “fixed” it? A recurrence after action items were marked done is the loudest possible signal that your fix addressed a symptom, not the cause — usually a root-cause-analysis quality problem worth revisiting. Track “recurrence after fixed” as a metric and it becomes your honest report card on whether the whole loop works.

A closing-the-loop checklist

For each postmortem finding:
- [ ] Resized to a shippable slice, not a stalled project
- [ ] In the shared backlog, owned, dated, prioritized against features
- [ ] Severity-appropriate priority (SEV-1 findings not silently deferred)
- [ ] Reviewed in a recurring forum until shipped
- [ ] Verified against the original failure, not just marked "done"
- [ ] Recurrence checked over the following weeks/months

Common mistakes

  • Confusing “merged” with “prevented.” The verification link exists because they’re different.
  • Reliability work in a separate, invisible backlog. Out of planning sight, out of mind.
  • Deferring silently. Deferral is fine if decided knowingly; fatal if it just happens.
  • No recurrence check. Without it, you never learn whether your fixes work — you just hope.

Wrapping up

A postmortem creates knowledge; closing the loop creates prevention. Resize findings into shippable slices, prioritize them honestly against feature work, review them until they ship, verify them against the original failure instead of trusting a “done” checkbox, and check for recurrence over time. The teams that do this stop meeting the same incident twice — which is the entire return on every hour they spend writing postmortems.

Newsletter

Free: the DevOps AI Incident-Triage Cheat Sheet

Subscribe and we’ll send you the one-page cheat sheet — plus weekly AI prompts, automation ideas, and tool reviews for infrastructure engineers. One email a week. No spam, unsubscribe anytime.

  • AI Incident-Triage Cheat Sheet (PDF)
  • Access to 2,778 DevOps AI prompts
  • One practical workflow email per week
Free download · 368-page PDF

Get 500 Battle-Tested DevOps AI Prompts — Free

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.