Standing Up an Incident Response Process From Scratch

Most teams don’t build an incident response process on purpose. They accumulate one by accident, one bad outage at a time, and end up with a pile of half-remembered conventions that live in three people’s heads. Then those three people leave, or they’re on vacation during the big one, and you discover you never actually had a process — you had a habit.

If you’re starting from nothing, or from that pile of habits, this is the minimum viable incident process I’d stand up first. It’s deliberately small. The goal isn’t a heavyweight framework; it’s a lightweight, written, agreed-upon set of defaults that turn “everyone panics differently” into “everyone does the same known thing.” You can build the fancy version later. Start here.

The five things you actually need

A working incident process needs exactly five pieces. Everything else is an optimization on top of these.

A way to declare — an agreed trigger and a place to do it.
Severity levels — so response scales to impact.
Roles — at minimum, someone who coordinates.
A comms plan — who gets told what, and how often.
A runbook — one page that ties it together and lives where people look.

Let’s build each one in its lightest useful form.

1. How to declare

Pick one channel where incidents get declared and lived in. A dedicated #incidents channel that spawns a per-incident channel is ideal, but even a single well-known channel beats “wherever someone happened to post.”

Write the trigger down: declare when customers are affected and it isn’t self-resolving, when you need to coordinate more than one person, when impact is growing with unknown cause, or when you’re simply unsure. Give every on-call engineer the authority to declare without asking permission, and make declaring blameless so nobody hesitates. The declaration template can be six lines:

🚨 DECLARING INCIDENT
What: [symptom, customer-facing]
Impact: [who/how many, since when]
Severity: [working SEVx]
Coordinating: [name]
Next update: [~15 min]

2. Severity levels

You need levels so a flaky dashboard and a checkout outage don’t get the same response. Keep it to four, anchored on customer impact — how many users, how badly — not on how scary the symptom looks.

SEV1  Core down / data loss / security. All-hands, page anyone,
      customer comms in minutes.
SEV2  Major degradation or key feature down, workaround exists.
      On-call + owning team now, status update.
SEV3  Limited impact, easy workaround, non-core. Business hours,
      owning team, no night page.
SEV4  Cosmetic / near-zero impact. Ticket, backlog.

The most important line in that table is the invisible one: SEV1 and SEV2 page humans at night; SEV3 and SEV4 wait for morning. Write that explicitly so the 2 AM responder never has to decide alone whether to wake someone. Default to declaring high and downgrading once you confirm impact is smaller.

3. Roles

The one role you cannot skip is a coordinator — call them the incident commander. Their job is to coordinate the response, not to fix the system. They hold the picture, assign work, decide severity, and keep comms flowing. On a small team the coordinator might also be the fixer at first, but the coordination job must be named so it doesn’t silently go undone.

As incidents get bigger, split off more roles: a communications owner for stakeholder and customer updates, a scribe to keep the timeline, and operations/subject-matter people who do the actual investigation. Don’t over-formalize on day one — a two-person incident needs a coordinator and a fixer, not a five-role org chart. But write down what each role does so that when the big one hits, you can staff it instantly.

4. A comms plan

Decide in advance who hears about incidents and how often, so nobody’s improvising stakeholder management mid-crisis. The minimum viable version is a table:

Audience        When notified          Channel        Cadence
--------------  ---------------------  -------------  ---------
Responders      On declare             #inc channel   Continuous
Internal team   SEV1/SEV2 on declare   #incidents     Every 15-30m
Leadership      SEV1 on declare        Direct + chan  Every 30m
Customers       Customer-impacting     Status page    As status
                                                       changes

The single highest-leverage comms habit is a cadence promise: “next update in 15 minutes even if nothing’s changed.” It stops anxious stakeholders from pinging responders every two minutes and buys the fixers quiet to work. Bake it into your process from day one.

5. The runbook that ties it together

All of the above is worthless if it lives in a doc nobody can find at 2 AM. Put it on one page, linked from your incident channel topic, your on-call tooling, and your team wiki’s front door. It should contain: the declaration trigger and template, the severity table with the paging line, the role definitions, the comms table, and a link to any service-specific runbooks. If a brand-new on-call engineer can open that one page during their first incident and know what to do, you’ve built the process. If they can’t find it, you haven’t.

A starter checklist to stand it all up

INCIDENT PROCESS — MINIMUM VIABLE SETUP

[ ] Create #incidents channel; document how to spin up
    a per-incident channel.
[ ] Write the declaration trigger + 6-line template.
[ ] Give all on-call engineers authority to declare.
[ ] Define SEV1-SEV4 by customer impact.
[ ] Write the explicit night-paging line (SEV1/2 = page now).
[ ] Name the incident commander role + first-5-minutes duties.
[ ] Write the comms table + cadence promise.
[ ] Put it ALL on one linked runbook page.
[ ] Run one gameday against a fake alert to test it.
[ ] Schedule a review after the first few real incidents.

Common mistakes when building from scratch

Over-engineering it. A twelve-role framework with five approval gates will be ignored under pressure. Start minimal; add ceremony only when a real incident proves you needed it.
Writing it but never testing it. A process that’s never been run in a gameday is a hypothesis, not a process. Rehearse it once before you rely on it.
Burying the runbook. If people can’t find it half-asleep, it doesn’t exist. One page, linked everywhere.
Making declaration heavy. If declaring feels like a big deal, people won’t do it in time. Keep it routine and blameless.
Never iterating. Your first process will have gaps. Review after the first handful of real incidents and fix what hurt.

Let the first real incidents teach you

The point of starting minimal is that real incidents will show you exactly where your process needs to grow — and they’ll show you far more accurately than any amount of up-front planning. After each of your first several incidents, spend five minutes asking: what did we improvise because the process didn’t cover it? Then add just that. A process built this way stays lean and stays used, because every piece of it earned its place in a real outage. An AI assistant can help here too: feed it your incident channel scrollback and have it flag the moments where people were confused about what to do, which points straight at the process gaps worth closing.

Wrapping up

You don’t need a heavyweight framework to respond to incidents well. You need five small things — a way to declare, severity levels, a coordinator role, a comms plan, and a one-page runbook — written down, agreed on, and findable at 2 AM. Build those, test them once against a fake alert, and let real incidents tell you what to add next. A lean process that everyone actually follows beats an elaborate one that lives in a wiki nobody opens.

Any AI assistance in building or running your process produces drafts for humans to review. People own the decisions.