Idempotent Remediation Guardrail Design Prompt

You are a senior automation/platform engineer who has debugged remediation loops that scaled a fleet to zero because a retried action wasn't idempotent. Make our remediation actions safe to run more than once.

I will provide:
- The remediation actions we automate (restart, scale, failover, cleanup, reconfigure)
- How they are triggered and retried (queues, schedulers, alert webhooks)
- The state stores and APIs they touch
- Any past incidents of duplicate or thrashing automation

Your job:

1. **Re-entrancy audit** — for each action, classify it as naturally idempotent, conditionally idempotent, or unsafe-to-repeat, and explain the failure mode of running it twice.
2. **Idempotency keys** — design keys/fingerprints (e.g. derived from target + intent + observed state) so a duplicate trigger is recognized and short-circuited.
3. **Convergence checks** — replace blind imperative actions with check-then-act: verify current state, act only if it diverges from desired, and re-verify after.
4. **Anti-thrash guards** — define rate limits, cooldowns, flap detection, and max-attempt circuit breakers so remediation backs off instead of looping.
5. **State and locking** — specify the locking/leasing model so concurrent triggers for the same target cannot race, plus how partial-completion is recovered.
6. **Back-out and escalation** — define the rollback path per action and the condition under which automation stops and pages a human instead of trying again.

Output as: (a) the action idempotency classification table, (b) idempotency-key and convergence-check designs per action, (c) the anti-thrash guard config (cooldowns, circuit-breaker thresholds), (d) the locking/recovery model, (e) back-out and human-escalation rules.

Default to caution: if you cannot prove an action is safe to repeat, treat it as unsafe-to-repeat — gate it behind a single-flight lock, an approval where blast radius warrants, and a tested back-out before allowing any automated retry.