📝 Build an AI Changelog Bot That Posts Merged-PR Summaries to Slack

The first time someone in my team asked “wait, what actually shipped this week?” I realized our changelog lived nowhere. It was scattered across PR titles, commit messages, and a few half-remembered Slack threads. So I built a bot that watches merged pull requests, asks a model to summarize them in plain language, and drops a tidy weekly changelog into our #releases channel. Here is how it came together, and where I made sure a human stayed in the loop.

Why a changelog bot beats raw PR titles

GitHub already emits events when a PR merges, and you could just forward those verbatim. But raw PR titles are written for reviewers, not readers. “fix: handle nil pointer in retry loop (#4821)” means nothing to the support engineer trying to figure out why a customer’s symptom disappeared. The job of a changelog is translation: from engineering shorthand to “we fixed a crash that happened when a request timed out and retried.” That translation is exactly the kind of bounded, low-stakes summarization work a model does well, as long as you treat it like a fast junior engineer whose output you skim before it ships.

Collecting the merged PRs

I run this on a weekly cron rather than reacting to every merge, which keeps the channel quiet. The collector hits the GitHub API for PRs merged since the last run:

async function fetchMergedPRs(repo, sinceISO) {
  const res = await fetch(
    `https://api.github.com/repos/${repo}/pulls?state=closed&sort=updated&direction=desc&per_page=100`,
    { headers: { Authorization: `Bearer ${process.env.GH_TOKEN}` } }
  );
  const prs = await res.json();
  return prs.filter(
    (p) => p.merged_at && new Date(p.merged_at) >= new Date(sinceISO)
  );
}

I keep the GitHub token in an environment variable and never let it travel anywhere near the model. The model only ever sees titles, bodies, and labels.

Drafting the summary with AI

I hand the model a compact list and ask for grouped, plain-language bullets. The prompt is deliberately strict about format so the output drops straight into Block Kit:

const prompt = `You are writing an internal changelog for a DevOps team.
Group these merged PRs into "Features", "Fixes", and "Internal".
One short plain-language bullet each. No PR numbers in the text.
Return JSON: { "features": [], "fixes": [], "internal": [] }.

PRs:
${prs.map((p) => `- [${p.labels.map((l) => l.name).join(",")}] ${p.title}`).join("\n")}`;

I iterate on this kind of prompt in a scratch workspace before wiring it into the bot. Our prompt workspace is where I test phrasings against real PR dumps, and the reusable versions live in prompts. If you want a starting library instead of writing from scratch, the prompt packs have changelog and release-note templates.

Pro Tip: Ask the model to return JSON and validate it before rendering. A changelog that occasionally produces a malformed bullet is annoying; a bot that crashes on a stray backtick is a pager event.

Rendering with Block Kit

Block Kit gives the changelog structure that a wall of text never will. I render each group as a section with a header:

function buildBlocks(changelog) {
  const blocks = [
    { type: "header", text: { type: "plain_text", text: "Weekly Changelog 📝" } },
  ];
  for (const [group, items] of Object.entries(changelog)) {
    if (!items.length) continue;
    blocks.push({
      type: "section",
      text: {
        type: "mrkdwn",
        text: `*${group[0].toUpperCase() + group.slice(1)}*\n` +
          items.map((i) => `• ${i}`).join("\n"),
      },
    });
  }
  return blocks;
}

Posting through Bolt

I use Bolt for Node because it handles signing-secret verification and the Web API client for me. The post itself is a chat.postMessage:

const { App } = require("@slack/bolt");
const app = new App({
  token: process.env.SLACK_BOT_TOKEN,
  signingSecret: process.env.SLACK_SIGNING_SECRET,
});

async function postChangelog(blocks) {
  await app.client.chat.postMessage({
    channel: "#releases",
    blocks,
    text: "Weekly changelog", // fallback for notifications
  });
}

The text fallback matters: it is what shows up in notifications and on screen readers when blocks cannot render. Skipping it is the most common Block Kit mistake I see.

Keeping a human in the loop

Here is the part I refuse to skip. The bot does not post directly to #releases. It posts a draft to a private #releases-draft channel with an “Approve” button. Someone reads it, fixes any summary that landed wrong, and clicks approve. Only then does it go public.

app.action("approve_changelog", async ({ ack, body, client }) => {
  await ack();
  await client.chat.postMessage({
    channel: "#releases",
    blocks: JSON.parse(body.actions[0].value),
    text: "Weekly changelog",
  });
});

This is the same principle I apply everywhere a model writes something customer-adjacent: AI drafts fast, a human reviews before it ships to a real audience. A wrong changelog bullet is not catastrophic, but it erodes trust in the channel, and trust is the only reason anyone reads it.

Verifying every inbound request

The “Approve” button sends an interaction payload back to my endpoint, and that endpoint is exposed to the internet. Bolt verifies the Slack signing secret automatically, but if you ever drop down to a raw handler, you must check the X-Slack-Signature and X-Slack-Request-Timestamp headers yourself and reject anything older than five minutes. Never trust an inbound webhook just because it claims to be from Slack.

Wiring it on a schedule

I run the whole flow from a small scheduled job. It fetches PRs, drafts the summary, posts the draft, and exits. The approval step is asynchronous and event-driven, so the cron job stays simple. For teams that already centralize this kind of automation, it slots neatly next to the other bots in your Slack ops toolkit.

If you want to build the prompt and Block Kit scaffolding faster, I lean on AI coding tools for the boilerplate. Cursor and GitHub Copilot are both good at generating the Bolt handler skeletons, and I use Claude for the summarization prompt itself.

Conclusion

A changelog bot is a small thing that quietly makes a team feel more informed. The pattern is simple: collect merged PRs, let a model translate them, render with Block Kit, and never post to the public channel without a human approving the draft. Keep your tokens out of the prompt, verify every inbound signature, and treat the AI like the eager junior it is. The result is a channel your team actually reads.