Dead-Man's-Switch and Automation Timeout Design Prompt
Add liveness and timeout safety to automated workflows — designing dead-man's switches, watchdog timers, stuck-run detection, and heartbeat alerts so an automation that hangs, stalls mid-action, or stops running entirely raises an alarm instead of failing silently.
- Target user
- Platform engineers running long-lived and scheduled automation
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a senior automation/platform engineer who has seen a scheduled job silently stop running for a month and a remediation worker hang mid-action holding a lock. Design liveness and timeout safety for our automation. I will provide: - The automated workflows (scheduled jobs, queue workers, long-running orchestrations) - Expected run cadence and per-step duration for each - The orchestration/scheduling stack we use - Past incidents of stuck, hung, or silently-stopped automation Your job: 1. **Liveness model** — for each workflow define what "alive and healthy" means: expected heartbeat cadence, max run duration, and max gap between successful runs. 2. **Dead-man's switches** — design switches that alert when an expected heartbeat or scheduled run does NOT arrive (catching the silent-stop case), not only when something errors. 3. **Per-step timeouts** — specify timeouts at action and overall-run level, with what happens on timeout (abort, release locks, mark for retry, escalate) so nothing hangs indefinitely holding resources. 4. **Stuck-run detection** — define how an in-progress run that's exceeded its budget is detected, force-terminated safely, and its partial work reconciled. 5. **Resource release** — ensure locks/leases/claims held by a killed run are released or expire, so a dead run can't block the fleet. 6. **Escalation** — map each timeout/liveness failure to the right alert and the human-handoff condition. Output as: (a) the per-workflow liveness spec (heartbeat, max-duration, max-gap), (b) the dead-man's-switch alert rules, (c) the timeout matrix (per-step and per-run with on-timeout behavior), (d) stuck-run detection and safe-termination procedure, (e) resource-release and escalation rules. Default to failing loud and safe: prefer aborting and alerting over letting a run hang, ensure a force-terminated run releases its locks and leaves no half-applied change, and require a tested back-out for any action that could be interrupted mid-flight.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Automation Retry-Budget and Timeout Topology Design Prompt
Design end-to-end timeout and retry budgets across a multi-hop automation chain (trigger to queue to worker to downstream API) so retries do not stack into retry storms, exceed the caller's deadline, or hammer a degraded dependency.
-
Scheduled Report Generation and Distribution Pipeline Design Prompt
Design a scheduled reporting pipeline that generates reports from a consistent data snapshot, distributes them reliably, and never sends a partial, stale, or duplicate report when a run retries or overlaps.
-
Webhook Ingest Async Queue Decoupling Design Prompt
Design a webhook ingest tier that acknowledges deliveries fast, persists the raw payload to a durable queue, and processes automation work asynchronously so slow downstream logic never causes sender retries or lost events.
-
Multi-Step Ops Workflow Checkpoint Orchestration Prompt
Orchestrate a long, multi-step operational workflow (migration, rollout, recovery) so it is restartable from durable checkpoints, compensates partial progress on failure, and never leaves the system in an unknown half-applied state when a step crashes mid-flight.
More Automation prompts & error guides
Browse every Automation prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.