🚦 Gating Slack App Manifests in CI: Catch Scope Creep and Config Drift on Every PR

I once approved a Slack app PR that added a single line to the manifest. The diff was three characters of context and one new entry under oauth_config.scopes: channels:history. I skimmed it, saw “small change,” approved. That one scope let the bot read the full message history of every public channel it was in — a meaningful expansion of what a compromised token could exfiltrate — and it sailed through review because the diff looked trivial and nobody had built a gate that treated scope changes as the security events they are.

This is a guide to building that gate. The Slack app manifest is a permissions document in config-file clothing, and it deserves the same CI rigor you’d apply to a Terraform plan: validate the schema, diff the scopes loudly, and check that the committed file actually matches the live app. Once you’ve version-controlled the manifest, these checks are what make that version control mean something.

Why the manifest needs a gate at all

The manifest defines your app’s OAuth scopes, event subscriptions, interactivity URLs, redirect URLs, and Socket Mode setting — the entire surface area of what the app can do and where it sends data. A change to any of these can expand the blast radius dramatically, and the dangerous changes are precisely the ones that look small. A new scope is one line. A widened redirect URL is one string. Nobody fears these in review the way they’d fear a hundred-line code change, which is exactly the problem.

There’s a second issue: if your app is editable in the Slack UI, someone can change the live app without touching the repo at all. Now your committed manifest is fiction — it describes an app that no longer exists. If you’re version-controlling your bot’s config, drift detection is what keeps that fiction from setting in.

Step one: validate the schema

Before anything else, fail the build if the manifest isn’t structurally valid. Slack provides apps.manifest.validate for exactly this:

curl -s -X POST https://slack.com/api/apps.manifest.validate \
  -H "Authorization: Bearer ${SLACK_CONFIG_TOKEN}" \
  -H "Content-Type: application/json" \
  --data "{\"manifest\": $(cat manifest.json)}" \
  | jq -e '.ok == true' > /dev/null || { echo "Manifest invalid"; exit 1; }

This catches structural mistakes — a malformed event subscription, an unknown field, a bad URL — before they reach a workspace. It’s cheap and it goes first.

Step two: diff the scopes and make additions loud

This is the heart of the gate. Compute the set of OAuth scopes added relative to the base branch, and surface every addition as something a human must consciously approve:

# Extract bot scopes from base and head, diff them
base_scopes=$(git show origin/main:manifest.json | jq -r '.oauth_config.scopes.bot[]' | sort)
head_scopes=$(jq -r '.oauth_config.scopes.bot[]' manifest.json | sort)

added=$(comm -13 <(echo "$base_scopes") <(echo "$head_scopes"))

if [ -n "$added" ]; then
  echo "::warning::New OAuth scopes added — REQUIRES SECURITY REVIEW:"
  echo "$added"
  # post these to the PR as a comment and require a 'scope-reviewed' label to merge
fi

The goal isn’t to block all scope additions — sometimes you genuinely need a new one. The goal is to make it impossible to add a scope accidentally and silently. Surface every addition in the PR, require an explicit acknowledgment (a label, a second reviewer, a comment), and the channels:history mistake I made becomes a deliberate, documented decision instead of a skim-and-approve.

Apply the same treatment to event subscriptions (settings.event_subscriptions.bot_events) and to any change in request, redirect, or interactivity URLs. A new event type means the app now receives data it didn’t before; a changed redirect URL is an OAuth-flow security concern. Both belong in the loud-diff bucket.

Step three: detect drift from the live app

Pull the live manifest with apps.manifest.export and diff it against the committed file. If they differ, someone edited in the UI and your repo is lying:

live=$(curl -s -X POST https://slack.com/api/apps.manifest.export \
  -H "Authorization: Bearer ${SLACK_CONFIG_TOKEN}" \
  --data "app_id=${SLACK_APP_ID}" | jq '.manifest')

if ! diff <(echo "$live" | jq -S .) <(jq -S . manifest.json) > /dev/null; then
  echo "::error::Live app manifest has drifted from the committed file."
  echo "Someone edited the app in the UI. Reconcile before merging."
  exit 1
fi

Run this on a schedule as well as on PRs — drift can happen any time, not just when someone opens a pull request. Catching it early keeps the manifest-as-code promise honest.

Encoding org policy as assertions

Beyond diffs, encode the rules your org actually cares about as hard assertions:

# No wildcard redirect URLs
jq -e '.oauth_config.redirect_urls // [] | all(contains("*") | not)' manifest.json \
  || { echo "Wildcard redirect URL is forbidden"; exit 1; }

# admin.* scopes require an explicit exception
if jq -e '.oauth_config.scopes.bot[]? | select(startswith("admin."))' manifest.json > /dev/null; then
  echo "::warning::admin.* scope present — requires exception label"
fi

These turn vague governance intentions (“we don’t allow wildcard redirects”) into a build that fails. The policy lives in code, gets reviewed like code, and applies uniformly.

Where AI fits

I had an LLM draft most of this pipeline, and it’s a good use of the tool — the jq incantations and the CI YAML are fiddly and well-trodden. A prompt like “write a GitHub Actions job that validates a Slack manifest, diffs added bot OAuth scopes against main, and comments them on the PR” gets you a working first pass.

Generate a GitHub Actions workflow that, on pull request, validates manifest.json via apps.manifest.validate, computes added bot OAuth scopes versus the base branch with jq and comm, and posts any additions as a PR comment. Fail the build on validation errors.

The AI drafts the plumbing; the human supplies the policy and reviews each flagged scope. That’s the right division — the model has no opinion about whether channels:history is acceptable for your bot, and it shouldn’t. Keep your refined versions in a prompt library alongside the related scope least-privilege audit prompt, so the next app you onboard starts from the same gate.

Wrapping Up

A Slack manifest is a permissions document, and the changes that expand your blast radius are the ones that look smallest in a diff. A CI gate fixes that by validating the schema, diffing scopes and URLs loudly enough that no addition is accidental, and checking that the live app hasn’t drifted from the file you reviewed. Let AI write the pipeline, encode your own policies as assertions, and keep a human on every flagged scope. Do it, and the three-character diff that quietly hands your bot the keys to every channel’s history becomes a deliberate decision — which is the only kind of decision a permission change should ever be.