SaltStack Reactor & Orchestrate Event-Driven Automation Prompt
Design a Salt reactor and orchestrate workflow that responds to events on the bus safely, with loop guards and idempotent orchestration instead of runaway reactions.
- Target user
- Infrastructure engineers running SaltStack at scale
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior SaltStack engineer who has watched a badly-written reactor turn a single minion event into a master meltdown, and designs event-driven automation defensively. I will provide: - The event I want to react to (custom event tag, minion start, [BEACON] like inotify/service, or a job-completion event). - What should happen in response (run a state, kick an orchestrate runner, notify, remediate). - The fleet size and which minions are in scope. Your job: 1. **Match the event precisely** — write the reactor SLS with a tag match tight enough that unrelated events never trigger it; show the exact event tag and the `data` fields you key on. 2. **Choose the right reaction type** — decide between `local` (run on the minion), `runner`, or `wheel`, and justify it; prefer `runner: state.orchestrate` for anything multi-step. 3. **Prevent reaction loops** — add a guard so a state change that itself emits an event can't trigger the same reactor recursively; show the loop scenario and the break condition. 4. **Make orchestration idempotent and ordered** — in the orchestrate SLS, use `require`/`onchanges` so steps run in order and re-running is safe; gate destructive steps behind a check. 5. **Bound the blast radius** — target the minimum set of minions; add `batch` for rolling actions and a concurrency limit so a fleet-wide event doesn't fan out to a thundering herd. 6. **Surface observability** — emit a structured event or log at start/end so reactions are auditable. Output as: (a) the reactor SLS, (b) the orchestrate SLS, (c) a sequence diagram (text) of event -> reactor -> orchestrate, (d) the loop/blast-radius risks and how each is mitigated. Do not enable the reactor in master config as part of this. Provide the config stanza separately and note that it should be tested with a single targeted minion before fleet-wide enablement.
Why this prompt works
SaltStack’s event bus is its superpower and its footgun. The reactor system lets you turn any event — a minion booting, a beacon firing on a config file change, a custom event from your app — into automated action across the fleet. But the same architecture makes a sloppy reactor catastrophic: a tag match that’s too loose fires on unrelated events, and a reaction that emits an event matching its own trigger creates an infinite loop that hammers the master. This prompt front-loads exactly those two failures by requiring a precise tag match and an explicit loop-break condition with the scenario spelled out.
The reaction-type decision is where experience shows. Beginners reach for local to run a state directly, which works for one minion and falls apart the moment the response needs multiple coordinated steps. By steering the model toward runner: state.orchestrate for anything multi-step and demanding require/onchanges ordering, the prompt produces orchestration that’s idempotent and re-runnable — the difference between a reaction you can trust to fire repeatedly and one that corrupts state on the second event. Bounding the blast radius with batch and concurrency limits addresses the thundering-herd problem that occurs when a fleet-wide event (like a mass minion restart) triggers the same reaction on thousands of nodes at once.
The off-switch safety note reflects a hard-won operational truth: auto-remediation reactors will eventually fight a human during an incident, restarting the service an engineer just stopped on purpose. Building in a pillar flag or grain to disable the reactor without editing master config keeps humans in control. Pair this with the SaltStack states & pillar design prompt and the GitOps for infrastructure prompt, and explore the IaC category for more configuration-management tooling.
Related prompts
-
GitOps for Infrastructure Prompt
Design a GitOps workflow that reconciles cloud and platform infrastructure (not just app manifests) from Git using Flux/Argo plus an IaC operator, with safe drift handling and promotion.
-
SaltStack States & Pillar Design Prompt
Design idempotent Salt state trees and a secure pillar/grains data layer — top files, environments, targeting, and reactor/orchestration — without the spaghetti most Salt deployments rot into.