# Disaster Recovery Runbook

Version: 1.0.0  
Last reviewed: 2026-07-10  
License: MIT

Implementation guide: https://steadyops.best/articles/ha-dr-runbooks/

## Document control

- Service:
- Environment:
- Owner:
- Last reviewed:
- Last drill:
- Next drill:
- Evidence location:

## Recovery contract

- Business impact:
- Severity:
- RPO:
- RTO:
- Incident commander:
- Recovery operator:
- Business validator:
- Communications owner:

## Trigger

Describe the measurable condition that activates this runbook.

## Stop conditions

Stop and escalate when any of the following are true:

- The recovery source cannot be verified.
- Replica lag or backup age exceeds the agreed RPO.
- The target recovery point is unclear.
- Writes may continue against more than one primary.
- Credentials, DNS, routing, or storage access are unavailable.
- A destructive step would remove evidence needed for investigation.

## Dependencies and recovery order

1. Identity, secrets, and network access.
2. Primary data store.
3. Queues and object storage.
4. APIs in controlled mode.
5. Workers after data consistency is confirmed.
6. Technical health validation.
7. Critical business transaction.
8. Normal traffic and background processing.

## Pre-recovery evidence

```bash
date -u
hostname -f
ip addr
systemctl --failed
```

Add database, cluster, routing, storage, and application checks for the real environment.

## Recovery source

- Source type:
- Backup/snapshot identifier:
- Base-backup timestamp:
- WAL/archive range:
- Checksum or integrity evidence:
- Encryption key/credential owner:

## Recovery procedure

For every step include command or action, expected result, stop condition, owner, timestamp, and evidence file.

### Step 1 — isolate unsafe writes

### Step 2 — verify recovery source

### Step 3 — restore or promote

### Step 4 — update routing

### Step 5 — reconnect dependencies

### Step 6 — validate technical health

### Step 7 — validate the business transaction

### Step 8 — resume traffic and background processing

## Communications

- Detection message:
- Customer-impact confirmation:
- Recovery started:
- Recovery progress interval:
- Service restored:
- Post-incident follow-up:

## Final evidence

- Recovered point:
- Measured data loss:
- Recovery start:
- Technical recovery:
- Business validation:
- Total recovery time:
- RPO met:
- RTO met:
- Remaining risks:
- Follow-up owners and deadlines: