aws
Most teams reach for Auto Scaling to handle Black Friday traffic. That is the least interesting thing it does. Configured with the right health checks and spread across Availability Zones, an Auto Scaling group is a self-healing system: it notices dead instances, replaces them, rebalances across AZs, and keeps your load balancer from sending traffic into a hole. This guide is about using that machinery for reliability, not just for peak load.
The three parts that work together
Reliability here is not one feature but three, cooperating:
- The load balancer (an Application or Network Load Balancer) spreads requests across healthy targets and stops sending traffic to ones that fail health checks.
- Health checks decide what "healthy" means. This is where most reliability wins and losses actually happen.
- The Auto Scaling group maintains a target number of instances, replaces unhealthy ones, and balances capacity across the AZs you configured.
When these are set up correctly, a failed instance is detected by the load balancer, drained, terminated, and replaced by the Auto Scaling group in a healthy AZ, all without a page. When they are set up carelessly, the same failure becomes an outage.
Health checks: the setting everyone gets wrong
By default, an Auto Scaling group uses EC2 status checks. Those only confirm the instance is running at the infrastructure level. They will happily report "healthy" while your application has crashed, deadlocked, or is returning 500s to every request.
The fix is one setting: change the group's health-check type to ELB. Now the load balancer's
health check (hitting a real endpoint like /healthz) decides what healthy means, and the group
replaces any instance the app has stopped serving on. Two supporting settings matter:
- Deregistration delay (connection draining) lets in-flight requests finish before an instance is removed, so scaling actions do not sever live connections.
- Slow start eases traffic onto new instances so they can warm caches and JIT before taking a full share, which prevents a thundering-herd of errors right after a scale-out.
Make the health endpoint honest: it should check the dependencies the request path actually needs (database, cache), but guard against a shared dependency failing every instance at once and triggering a full-group replacement storm.
Spread across AZs, and keep headroom
An Auto Scaling group configured across two or three Availability Zones gets failure absorption for free. AWS balances capacity across the AZs, and if one degrades, the group relaunches the lost instances in a healthy AZ. This is the mechanism that turns AZ-level events, the most common serious failure class, into non-events, as the Multi-AZ vs Multi-Region guide details.
The catch is headroom. If three AZs each carry a third of your traffic and one fails, the surviving two must absorb 50 percent more each, immediately, before replacement capacity launches. Size your minimum and desired capacity so an AZ loss does not saturate the group. The math is simple and the consequence of ignoring it is a cascading failure.
| Setting | Weak default | Reliable configuration |
|---|---|---|
| Health-check type | EC2 status only | ELB target health on a real endpoint |
| AZ coverage | Single AZ | 2 to 3 AZs, balanced |
| Deregistration delay | 0s (abrupt) | Long enough to drain in-flight requests |
| Capacity headroom | Sized for steady state | Survives one AZ loss at peak |
Where Auto Scaling stops
Auto Scaling is a regional service. It handles instance failures and AZ failures automatically, but it depends on the region's control plane to launch instances. During a regional control-plane outage, exactly the scenario in the December 2021 event, new launches can fail. If your recovery plan assumes the group will scale up mid-incident, it may not. This is why the DR checklist warns against satisfying a tight recovery target with a strategy that needs new capacity, and why a full region event still requires a second region.
Prove it with a game day
None of this is real until you have watched it work. In a controlled window, terminate instances and simulate an AZ impairment, then confirm the group relaunches capacity in a healthy AZ, the load balancer stops routing to the failures, and users see no errors. Measure how long recovery takes and whether headroom held. Pair the test with external detection so you know when something breaks in production: independent monitoring plus instant outage alerts closes the gap between a failure starting and you noticing it.
For the broader reliability picture, the research at clouddowntime.com/data tracks how AWS actually fails, and if a provider incident breaches an SLA you may be owed credits, which you can check at cloudslacredit.com. Teams that want detection wired straight into their stack use Next Signal (sponsor) to catch provider events early.