Python Process Watchdog and Auto-Restart Prompt
Build a Python watchdog that monitors a long-running process or endpoint and restarts it with backoff when it hangs, crashes, or fails a health probe.
- Target user
- Platform engineers keeping unsupervised jobs and daemons alive
- Difficulty
- Intermediate
- Tools
- Claude, Cursor
The prompt
You are a senior reliability engineer who writes supervisor tooling that must never make an outage worse by flapping a service. I will provide: - What to watch (a PID, a command to (re)launch, or an HTTP health URL) - The failure signals that count as "unhealthy" - Restart limits and any maintenance windows Your job: 1. **Define health** — implement a single `is_healthy()` that checks the process and/or probes the endpoint with a timeout; never block forever. 2. **Supervise** — loop on a fixed interval, but isolate each check in `try/except` so a transient error does not kill the watchdog itself. 3. **Back off** — on repeated failures, restart with exponential backoff plus jitter, and enforce a max-restarts-per-window circuit breaker that stops flapping. 4. **Restart safely** — terminate the old process with SIGTERM, wait, then SIGKILL only if needed; confirm the new PID is actually healthy before counting success. 5. **Add a dry-run** — `--dry-run` logs intended restarts without spawning anything. 6. **Log structured events** — emit JSON lines (timestamp, state, action, attempt) for ingestion. 7. **Shut down cleanly** — handle SIGINT/SIGTERM to stop supervising and optionally drain the child. Output as: (a) the full script with type hints, (b) a config example, (c) a systemd unit to run the watchdog. Cap restarts and trip the breaker; an infinite restart loop is worse than a clean failure.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Asyncio TaskGroup Structured-Concurrency Python Script Prompt
Write a Python 3.11+ automation script that fans work out with asyncio.TaskGroup so failures cancel siblings cleanly, errors aggregate into an ExceptionGroup, and no task is ever silently orphaned.
-
Convert a Fragile Bash Pipeline to Python Prompt
Rewrite an overgrown, fragile Bash script full of pipes, subshells, and string parsing into a maintainable Python program with proper error handling, types, and tests.
-
Add Pytest Coverage to an Automation Script Prompt
Wrap an existing Python automation script in a pytest suite that mocks subprocess, network, filesystem, and clock side effects so the logic is tested fast and deterministically without touching real systems.
-
Idempotent Bulk File Rename & Migrate Tool Prompt
Write a Python tool that safely renames or relocates large batches of files using a rule set, with mandatory dry-run, collision detection, atomic moves, and an undo manifest so a bad run is reversible.
More Bash & Python Automation prompts & error guides
Browse every Bash & Python Automation prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.