🚨 Postmortem Anti-Pattern: Severity Mislabeling and Inconsistent SEV Levels

Overview

Severity is the dial that controls almost everything downstream of an incident: how many people get paged, how fast the response escalates, whether leadership is notified, whether a postmortem is even required, and how the incident is counted in reliability metrics. Severity mislabeling — assigning the wrong level, or applying the levels inconsistently from one incident to the next — quietly distorts all of it. A genuinely customer-impacting outage tagged SEV3 gets a lazy response and no review; a minor internal blip tagged SEV1 burns out the on-call and clutters the metrics. When two similar incidents get two different severities depending on who was on call, the SEV level stops meaning anything at all.

This anti-pattern matters because severity is a filter on the entire learning process. In most organizations, only incidents above a severity threshold get a full postmortem — so mislabeling doesn’t just misprioritize the response, it determines which failures the organization ever examines. Systematically under-labeling means real problems slip below the review line and are never analyzed; systematically over-labeling floods the process with noise until people stop taking severity seriously. And because SEV levels feed reliability dashboards and executive reporting, inconsistent labeling corrupts the numbers leadership uses to make investment decisions. Getting severity right — and consistent — is a prerequisite for every other part of incident management working.

Symptoms

• Two similar incidents, two different SEV levels — the same customer impact labeled SEV2 one week and SEV3 the next.
• Severity that tracks the responder’s mood — anxious on-calls over-escalate, tired ones under-escalate, with no objective anchor.
• Real customer impact tagged low to avoid the paperwork, escalation, or executive attention a higher SEV triggers.
• Minor issues tagged high out of caution or to get more hands, cluttering the metrics with false majors.
• Severity never revised as an incident grows or shrinks — set once at declaration and never updated.
• Arguments about severity during the incident because the criteria are ambiguous or absent.
• Incidents that should have had postmortems but didn’t because they were under-labeled below the review threshold.
• Reliability metrics that don’t match lived experience — the dashboard says a quiet quarter while the team remembers firefighting.

Common Root Causes

• Vague or missing severity definitions. If SEV levels aren’t defined by concrete, measurable criteria, everyone interprets them differently.
• Impact-based criteria absent. Severity gets set on gut feel about “how bad it seems” rather than on customer impact, scope, and duration.
• Incentives to under-report. Higher severities trigger executive visibility, blameful scrutiny, or more process, so teams quietly down-tag to avoid the friction.
• Incentives to over-report. In some cultures a high SEV is the only way to get enough responders fast, so people inflate to summon help.
• Set-and-forget declaration. Severity is assigned in the first chaotic minutes on incomplete information and never re-evaluated as the picture clarifies.
• No calibration across teams. Each team develops its own private sense of what SEV2 means, so severity isn’t comparable org-wide.
• The postmortem threshold creates a cliff. When “SEV2 and above get a review,” the level right below becomes a place to hide incidents from scrutiny.

Diagnostic Workflow

Severity should be a function of observable facts, not feelings. The core diagnostic is to re-derive the severity from an explicit, impact-based rubric and compare it to the label that was actually assigned.

Impact-based severity rubric (illustrative — define yours concretely):
  SEV1  widespread customer-facing outage; core function down;
        revenue/safety impact; all-hands, exec-notified
  SEV2  significant impact; major feature degraded OR subset of
        customers fully down; on-call + lead engaged
  SEV3  limited/internal impact; workaround exists; minor degradation;
        handled in business hours
  Dimensions to score: customer scope | function criticality |
                       duration | data/revenue/safety | workaround?

For any incident, score it against the rubric and check for drift between the derived level and the assigned one:

Severity consistency check:
  1. Score the incident on each dimension from the rubric.
  2. Derive the severity the rubric implies.
  3. Compare to what was actually assigned.
     derived > assigned -> UNDER-labeled (why? avoiding process/scrutiny?)
     derived < assigned -> OVER-labeled  (why? summoning help/caution?)
  4. Find 2-3 PAST incidents with similar impact. Same SEV?
     different SEV for same impact -> inconsistency confirmed.

Finally, examine the incentives, because most mislabeling is rational under a bad incentive. Ask: does a higher SEV bring help, or bring blame? Does the postmortem threshold create a cliff people down-tag to duck under? Consistent severity requires removing the reasons people have to game it.

Example Root Cause Analysis

Incident: A payment-processing endpoint failed for roughly 20% of transactions for 35 minutes. The on-call, wanting to avoid an executive page and the mandatory postmortem, tagged it SEV3.

Mislabeled version: “SEV3 — payment endpoint had elevated errors, resolved in 35 min. No postmortem required (below SEV2 threshold).”

Why the label is wrong, by the rubric:

1. Customer scope: 20% of paying customers could not complete purchases — that is “significant impact to a subset,” squarely SEV2 territory, not the “limited/internal” of SEV3.
2. Function criticality: payments is a core revenue function; failures there rank higher than the same failure in a peripheral feature.
3. Revenue impact: 35 minutes of 20% payment failure has direct, measurable revenue loss — a SEV1/SEV2 dimension the SEV3 label ignores entirely.
4. No workaround: affected customers had no alternate path to pay, which the rubric weights toward higher severity.

The consequence of the mislabel: because SEV3 sits below the postmortem threshold, this incident would have received no review at all. A recurring, revenue-impacting payment failure would go unanalyzed, and the reliability dashboard would under-count payment incidents — misinforming exactly the investment decisions that could fix it.

Consistency cross-check: two prior payment incidents of similar scope were tagged SEV2. Same impact, different label — inconsistency confirmed, and the difference is explained by the on-call’s incentive to avoid scrutiny, not by any real difference in the incident.

Corrected labeling: “SEV2 — payment endpoint failing for ~20% of transactions, 35 min, direct revenue impact, no customer workaround. Postmortem required.” And the deeper finding for the postmortem itself: the severity rubric permitted a revenue-impacting payment outage to be self-tagged below the review line, and the incentive structure rewarded doing so.

Prevention Best Practices

• Define severity by concrete, measurable criteria. Anchor each level in customer scope, function criticality, duration, and revenue/safety impact — not in how bad it feels.
• Make severity impact-based, not effort-based. Severity describes what customers experience, not how hard the fix was or how stressed the responder felt.
• Re-evaluate severity as incidents evolve. Set an initial level fast, but revise up or down explicitly as the real scope becomes clear; severity is a live value, not a declaration.
• Decouple severity from punishment. Ensure a higher SEV brings more help and attention, never blame, so nobody has a reason to down-tag to hide.
• Calibrate across teams regularly. Review past incidents together and check that the same impact gets the same label everywhere, so SEV levels are comparable org-wide.
• Soften the postmortem-threshold cliff. Avoid a hard line people game by allowing reviews for any incident that warrants one and spot-checking under-labeled incidents.
• Audit severity assignments periodically. Sample closed incidents, re-derive severity from the rubric, and correct systematic drift before it corrupts the metrics.

Quick Reference

# Severity mislabeling = wrong/inconsistent SEV -> distorts response, metrics, reviews.

Severity is a FUNCTION OF IMPACT, not effort or mood:
  dimensions: customer scope | function criticality | duration
              | revenue/data/safety | workaround exists?

Consistency check:
  1. score incident on rubric dimensions
  2. derive implied severity
  3. compare to assigned:
       derived > assigned -> UNDER-labeled (hiding from process?)
       derived < assigned -> OVER-labeled  (summoning help/caution?)
  4. same-impact past incidents get same SEV? if not -> inconsistent

Watch the incentives:
- higher SEV brings blame  -> people under-tag  -> decouple SEV from punishment
- higher SEV brings help   -> people over-tag   -> fix escalation, not severity
- postmortem threshold cliff -> down-tagging to duck -> spot-check below the line

Conclusion

Severity is the control dial for the entire incident lifecycle, and when it is mislabeled or applied inconsistently, everything it governs goes wrong at once: responses are misprioritized, on-calls are misdirected, metrics misreport reality, and — most damagingly — the wrong incidents get filtered out of the review process entirely, so the failures most worth learning from are the ones never examined. The fix is to make severity objective and consistent: define each level by measurable customer-impact criteria, re-derive it from that rubric rather than from feeling, revise it as incidents evolve, and calibrate across teams so the same impact always earns the same label. Above all, remove the incentives that make gaming severity rational — decouple higher SEVs from blame and soften the review-threshold cliff. A severity level should be a reliable statement about what customers experienced, so that the organization prioritizes, measures, and learns from its incidents based on what actually happened rather than on who happened to be holding the pager.