Recovery Validation Prompt: Gradual, Verified Restore After Mitigation
Turn 'the fix is in' into a staged, verified traffic-restoration plan with explicit health gates and abort criteria — so service comes back cleanly the first time instead of a premature all-clear that reopens the incident and doubles MTTR.
- Target user
- Incident commanders and SREs bringing a service back after mitigation
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are an SRE planning the recovery of a service after a mitigation has been applied. The goal is to restore full service in a staged, verified way so it comes back for good — not a premature all-clear that reopens the incident and inflates MTTR with a second round. Give me: - What was done: [THE MITIGATION APPLIED — rollback, failover, scale-up, flag change] - Current state: [WHAT % OF TRAFFIC IS SERVED, WHAT'S STILL DEGRADED OR DRAINED] - Health signals available: [KEY METRICS: ERROR RATE, LATENCY, SATURATION, QUEUE DEPTH] - Constraints: [PEAK/OFF-PEAK, DEPENDENCIES STILL RECOVERING, CACHES COLD] Work through this: 1. **Define "recovered."** State the concrete, measurable conditions that mean the service is actually healthy — not just "errors stopped," but the specific metric values sustained for a specific duration. Vague success criteria are how premature all-clears happen. 2. **Stage the restore.** Propose a ramp (e.g. 10% → 25% → 50% → 100%, or region by region), with a bake time at each stage long enough for lagging signals — cold caches, warming pools, downstream backpressure — to show up. 3. **Set a health gate per stage.** For each stage, the exact signals to watch and the pass thresholds required before advancing. Include the slow-to-appear signals that a hasty ramp skips past. 4. **Predefine abort criteria.** For each stage, the specific signal and threshold that means STOP and roll the ramp back one stage. Decide this before ramping, not during. 5. **Confirm the all-clear.** State what must hold, and for how long, before declaring the incident resolved — and what to keep watching after, since some relapses appear only under sustained full load. Output format: a "RESTORE PLAN" — a definition of RECOVERED, then a staged table (STAGE, TRAFFIC, BAKE TIME, HEALTH GATE, ABORT CRITERION) — and an ALL-CLEAR CONDITION plus POST-RECOVERY WATCH. Note which signals lag. Recommend the plan; leave every ramp step and gate decision to the human.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Why this prompt works
MTTR is not just time-to-mitigate — it is time until the incident is genuinely over, and a premature all-clear is one of the sneakiest ways to blow it. A team stops the bleeding, immediately flips everything back to 100%, declares victory, and then the service relapses under full load or cold caches and the whole incident reopens. The second round often runs longer than the first because now the team is tired and the timeline is muddied.
This prompt treats recovery as a first-class phase with the same rigor as mitigation. It forces a concrete, measurable definition of “recovered” up front — the antidote to “errors stopped, we’re good” — and stages the restore with bake times long enough for the lagging signals (warming pools, downstream backpressure, cold caches) that a hasty ramp jumps straight past. Each stage gets a pass gate, so restoration advances on evidence rather than optimism.
The decisive discipline is predefining abort criteria per stage before ramping begins. In the moment, a partial relapse triggers paralysis — is it really failing? — and the delay becomes a full second outage. Deciding the stop-and-revert threshold in advance removes that hesitation. Paired with keeping a human on every go/no-go gate, this turns a risky snap-back into a clean, one-time recovery, which is exactly where the “second incident” MTTR tax disappears.
Related prompts
-
Customer-Impact Quantifier Prompt
Turn raw incident signals into a defensible estimate of how many users, which segments, and how much revenue are affected — so severity, comms, and prioritization are sized on impact instead of gut feel, and the response matches the real blast radius.
-
Game-Day Scenario Generator Prompt
Generate realistic failure-injection scenarios and a facilitation script to rehearse incident response on your actual stack — so the team's detection, triage, and mitigation muscles are warm before a real outage, cutting MTTR when it counts.
-
Mitigate-Now vs. Keep-Diagnosing Decision Prompt
In the middle of a live incident, decide whether to apply an available mitigation immediately or keep diagnosing for root cause — so you stop the customer bleeding at the earliest safe moment instead of chasing 'why' while the clock runs, cutting time-to-restore.
-
War-Room Role Assignment Prompt
When a major incident pulls a crowd into the bridge, assign clear roles — commander, comms, scribe, ops leads — from who is actually present, so coordination overhead stops eating MTTR and every workstream has a named owner instead of five people debugging the same thing.
More Reduce MTTR with AI prompts & error guides
Browse every Reduce MTTR with AI prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.