Every post-mortem in our archive ends the same way: the teams that recovered fast had done unglamorous preparation work months earlier. This checklist is that work, ordered so each step makes the next one honest. Work through it top to bottom; the five steps also emit a machine-readable plan of record your auditors will love.
Step 1 - Map your real dependencies
You cannot recover what you don't know you depend on. For each workload, walk the request path and list every AWS service, every region, and every third party it touches - then do it again for the deploy path and the operate path (CI, container registry, secrets, DNS, feature flags, observability). Hidden dependencies are the recurring villain of major incidents: in the October 2025 event, teams "not using DynamoDB" went down with it via their auth provider, their flag vendor, or AWS's own control planes.
Exercise that works: in a staging copy, block one dependency at the network level and watch what actually breaks. Repeat monthly with a different dependency.
Step 2 - Set RTO and RPO with business sign-off
RTO (how long you can be down) and RPO (how much data you can lose) are business decisions wearing engineering clothes. Force real numbers per workload, and tier ruthlessly: the payment path might justify minutes/zero; the analytics warehouse can take a day. Without tiers, everything gets gold-plated and nothing gets tested.
Step 3 - Choose the cheapest strategy that meets each tier
Map tiers onto the standard four strategies (costed in Multi-AZ vs Multi-Region): backup-and-restore, pilot light, warm standby, active-active. Two rules keep this honest: never pick a strategy whose failover you won't test, and never satisfy a minutes-level RTO with a strategy that requires launching new capacity - the December 2021 event proved the control plane may be exactly what's down.
Step 4 - Backups, access, and the break-glass path
- Backups: automated, cross-region (and ideally cross-account, against credential compromise), with a restore tested monthly. Time the restore - that number is your real RPO-to-RTO bridge.
- Operator access: regional STS endpoints, pre-issued break-glass credentials in a sealed location, and at least one admin path that does not traverse the primary region or your SSO.
- Communication: a status page and incident channel hosted off your own infrastructure (the 2017 S3 outage took AWS's own status icons down with it).
- Detection: you can't fail over to a disaster you haven't noticed. External monitoring plus instant outage alerts beats waiting for customer tickets by 20–40 minutes on average.
Step 5 - Game days: measure, don't hope
Quarterly, per critical tier, simulate one of the three real failure classes:
- AZ loss - kill an AZ's worth of capacity; watch load balancing, data-store failover, quorum.
- Regional control-plane outage - freeze all launches/scaling/console access for 4 hours (policy-enforced); can you still serve, deploy a hotfix, rotate a credential?
- DNS/endpoint failure - blackhole one service endpoint; verify timeouts, retries, and circuit breakers behave.
Record achieved RTO/RPO against targets, file the gaps as engineering work with owners, and re-run. A DR plan that has never missed its targets has never been tested hard enough.