Your Multi-AZ Architecture Probably Has a Single Point of Failure

Cloud networking failures rarely look like failures at first.

Health checks are green. Dashboards look normal. Everything appears “highly available.”

Until one small dependency fails — and suddenly the architecture you thought was resilient isn’t.

Welcome to Failover Weekly, a short field guide on cloud networking and resilience, failure patterns, and the architectural lessons outages teach us.

Over the past years I’ve reviewed hundreds of cloud architectures across multiple organizations. One thing keeps showing up: systems designed for high availability that still contain hidden single-AZ dependencies.

This newsletter is about finding those before they find you.

Let’s start with one of the most common.

---

The NAT Gateway trap

You deployed across three Availability Zones. Your health checks are green. You sleep well at night.

You shouldn’t.

One of the most frequent resilience gaps I see in otherwise well-designed architectures involves NAT Gateways.

In AWS, NAT Gateways are Availability Zone–scoped resources. Each NAT Gateway lives inside a single AZ.

If private subnets in multiple AZs route their outbound traffic through a single NAT Gateway in one AZ, a failure of that AZ removes outbound connectivity for every subnet depending on it.

Your instances remain running. But they can no longer reach the internet or external services.

This affects things like: pulling container images, accessing third-party APIs, reaching package repositories, and downloading updates.

The compute layer survives. The architecture still fails.

---

The resilient pattern

The established design is straightforward:

Deploy one NAT Gateway per Availability Zone and ensure each private subnet routes to the NAT Gateway in the same AZ.

Example:

AZ-A private subnets → NAT-A
AZ-B private subnets → NAT-B
AZ-C private subnets → NAT-C

This provides two benefits: outbound connectivity survives an AZ failure, and traffic stays local to the AZ, avoiding cross-AZ data transfer.

One NAT Gateway per Availability Zone: each private subnet routes to the NAT Gateway in the same AZ

As of November 2025, AWS also offers Regional NAT Gateways — a single gateway that automatically expands across AZs based on workload presence. This simplifies management while maintaining high availability. Both approaches are valid; the one-per-AZ pattern gives you explicit control, while the regional mode reduces operational overhead.

---

NAT Gateways aren’t the only hidden dependency

When reviewing architectures, these four patterns show up repeatedly:

NLB cross-zone load balancing — For Network Load Balancers, cross-zone load balancing is off by default. Many teams leave it disabled for cost or isolation reasons without fully understanding the impact on traffic distribution when targets are unevenly distributed across AZs. This can lead to overloaded targets in some zones and idle capacity in others.

EBS volumes bound to a single AZ — Amazon EBS volumes exist in one AZ. Without snapshots, replication, or architectural redundancy, losing the AZ means losing the volume.

Incomplete RDS subnet groups — Amazon RDS failover depends on the subnet group configuration. Multi-AZ DB instances require subnets in at least two AZs; Multi-AZ DB clusters require at least three. If the subnet group does not include enough AZ coverage, failover cannot occur.

Automation that depends on the control plane — Some failover scripts rely on the EC2 API to modify infrastructure during an incident. But control-plane services may themselves be degraded during large-scale events, preventing those scripts from executing. AWS Well-Architected explicitly recommends relying on data-plane operations, not the control plane, during recovery.

---

One thing to do this week

Open your architecture diagram and ask one question for every component:

If one Availability Zone disappears right now, does this system continue working without manual intervention?

If the answer is “I think so” — you haven’t tested it. And if you haven’t tested it, it doesn’t work.

---

See you in the next issue. Next time we’ll look at a different hidden dependency: why DNS architectures often create larger blast radii than the systems they front.