🌅 AI-Generated On-Call Handoff Summaries in Slack

I have written some genuinely bad handoff summaries. Not from laziness — from exhaustion. After a twelve-hour on-call shift where two things broke and a third nearly did, the last thing my brain wants to do is reconstruct a coherent narrative of what happened, what’s still open, and what the next person needs to watch. So my handoffs would come out as a terse, half-remembered list: “DB thing — see thread. Disk alert, probably fine. Deploy paused, ask Sam.” The next engineer would inherit my fatigue along with my incidents.

The handoff is exactly the kind of synthesis task that an LLM is good at and a tired human is bad at. Pull the open incidents and the relevant Slack threads, summarize them into something readable, format it nicely, and — this is the part that keeps it safe — let the outgoing engineer read and edit the draft before it posts. The AI writes the first pass. The human, who actually knows what happened, signs off. That division of labor is the whole idea.

What a good handoff contains

Before automating it, it helps to name what you’re actually producing. A useful handoff summary has: the open incidents with current status, anything that’s degraded-but-not-paging, in-flight changes (deploys, migrations, paused pipelines), and explicit “watch this” items for the incoming engineer. The AI’s job is to assemble those from scattered sources into a clean, scannable message. Your job is to correct the things it inferred wrong — because it will infer some things wrong, the way a quick junior engineer summarizing a meeting they half-attended would.

Pulling the raw material

The model can only summarize what you feed it. So step one is gathering open incidents and the Slack threads attached to them. If you’re tracking incidents in a system with an API, pull the open ones; for the Slack side, fetch the threads from your incident channel using conversations.replies.

import os
from slack_bolt import App

app = App(
    token=os.environ["SLACK_BOT_TOKEN"],
    signing_secret=os.environ["SLACK_SIGNING_SECRET"],
)


def gather_threads(channel_id, thread_ts_list):
    """Pull the replies for each tracked incident thread."""
    threads = []
    for ts in thread_ts_list:
        resp = app.client.conversations_replies(channel=channel_id, ts=ts)
        messages = [
            {"user": m.get("user", "system"), "text": m.get("text", "")}
            for m in resp["messages"]
        ]
        threads.append({"thread_ts": ts, "messages": messages})
    return threads

A note on what you collect: pull the conversation content, not your bot tokens, API keys, or any infra credentials that might be floating in a thread. If an engineer pasted a connection string into an incident channel mid-debug (it happens), strip it before it reaches the model. The model never needs your secrets to write a summary, and it should never see them.

Summarizing with the model

Now hand the gathered material to Claude and ask for a structured handoff. I use a structured-output schema so I get back something I can render directly into Block Kit rather than free-floating prose I’d have to parse.

import json
from anthropic import Anthropic

anthropic = Anthropic()  # ANTHROPIC_API_KEY from the environment

HANDOFF_SCHEMA = {
    "type": "object",
    "properties": {
        "open_incidents": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "title": {"type": "string"},
                    "status": {"type": "string"},
                    "next_step": {"type": "string"},
                },
                "required": ["title", "status", "next_step"],
                "additionalProperties": False,
            },
        },
        "watch_items": {"type": "array", "items": {"type": "string"}},
        "in_flight_changes": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["open_incidents", "watch_items", "in_flight_changes"],
    "additionalProperties": False,
}


def draft_handoff(threads):
    response = anthropic.messages.create(
        model="claude-opus-4-8",
        max_tokens=2048,
        thinking={"type": "adaptive"},
        system=(
            "You write end-of-shift on-call handoff summaries for an SRE team. "
            "Summarize open incidents, in-flight changes, and items the next "
            "engineer should watch. Be precise and conservative — do not invent "
            "status you cannot see in the threads. Flag anything ambiguous so a "
            "human can correct it before this is posted."
        ),
        output_config={"format": {"type": "json_schema", "schema": HANDOFF_SCHEMA}},
        messages=[
            {"role": "user", "content": json.dumps(threads)},
        ],
    )
    return json.loads(response.content[0].text)

The system prompt does real work here. “Do not invent status you cannot see” is a direct counter to the LLM’s tendency to smooth over gaps with plausible-sounding fabrication — the single most dangerous failure mode for a handoff, because a confidently-wrong “resolved” can leave a live incident unattended. Telling it to flag ambiguity instead of guessing turns those gaps into review prompts for the human rather than silent errors.

Formatting as Block Kit

Render the structured draft into a Block Kit message. Because the data is already typed, this is mechanical — no parsing, no guessing.

def build_handoff_blocks(handoff):
    blocks = [
        {"type": "header", "text": {"type": "plain_text", "text": "🌅 On-Call Handoff"}},
    ]
    for inc in handoff["open_incidents"]:
        blocks.append({
            "type": "section",
            "text": {
                "type": "mrkdwn",
                "text": (
                    f"*{inc['title']}* — {inc['status']}\n"
                    f"› _Next:_ {inc['next_step']}"
                ),
            },
        })
    if handoff["watch_items"]:
        blocks.append({"type": "divider"})
        blocks.append({
            "type": "section",
            "text": {
                "type": "mrkdwn",
                "text": "*Watch:*\n" + "\n".join(f"• {w}" for w in handoff["watch_items"]),
            },
        })
    return blocks

The review-before-send step

This is the part that makes the whole thing safe to run. The bot does not post the handoff to the team channel directly. It opens the draft in a modal (or DMs it to the outgoing engineer) with an editable text area and a “Looks good — post it” button. The engineer reads what the model wrote, fixes the one incident it mislabeled, deletes the “watch” item that’s actually resolved, and then it goes out.

@app.action("edit_handoff")
def open_handoff_modal(ack, body, client):
    ack()
    client.views_open(
        trigger_id=body["trigger_id"],
        view={
            "type": "modal",
            "callback_id": "handoff_review",
            "title": {"type": "plain_text", "text": "Review Handoff"},
            "submit": {"type": "plain_text", "text": "Post to #oncall"},
            "blocks": [
                {
                    "type": "input",
                    "block_id": "handoff_text",
                    "label": {"type": "plain_text", "text": "Edit before posting"},
                    "element": {
                        "type": "plain_text_input",
                        "multiline": True,
                        "action_id": "content",
                        "initial_value": body["message"]["text"],
                    },
                }
            ],
        },
    )

The modal is the human-in-the-loop boundary made concrete. The AI is the fast junior who drafts the shift report in seconds; the senior — the person who actually lived the shift — reads it, corrects it, and owns the version that ships. A handoff that posts without a human reading it is just an unreviewed bot guess wearing a tie.

Pro Tip: Pre-fill the modal but keep the “post” action gated behind the submit. If the engineer closes the modal without submitting, nothing posts — the safe default is silence, not an unreviewed summary in the team channel.

Don’t skip signature verification

The interactive parts — the “edit” button, the modal submission — all arrive as Slack interaction payloads to your endpoint. Verify them. Slack signs each with HMAC SHA256 over the timestamp and raw body using your signing secret, delivered in the x-slack-signature header alongside x-slack-request-timestamp. Bolt verifies these for you when configured with your signing_secret, but if you’ve built a custom receiver, that verification is non-negotiable. An unverified interaction endpoint lets anyone trigger your “post to #oncall” action. The same boundary discipline as everywhere else: prove the request is real before you act on it.

Fitting it into the workflow

The trigger can be a scheduled job near shift-end, a slash command the engineer runs when they’re ready, or a button in a daily ops message. Whichever you choose, keep the sequence fixed: gather → summarize → review → post. If you’re already running structured ops tooling, this slots in alongside your incident-response and code-review flows, and you can reuse the same Slack app and signing secret across them. For iterating on the summarization prompt itself, a saved prompt library or downloadable prompt packs save you from re-tuning the system prompt every shift.

Wrapping Up

On-call handoffs are a synthesis task that tired humans do poorly and LLMs do well — which makes them an almost ideal place to delegate the first draft. Gather the open incidents and threads, let the model summarize into structured, reviewable output, render it as Block Kit, and put the outgoing engineer between the draft and the post. Verify every inbound signature, keep secrets out of the model’s context entirely, and treat the AI as the fast junior it is: it writes the shift report, the human who lived the shift signs off. The next person inherits a clean handoff instead of your 11 PM exhaustion.