All issues 25

Failover Weekly No. 8

386 Outages in One Week

The Network You Do Not Own Can Still Fail Your Workload

386 Outages in One Week: diagram

On April 25, Zayo Group experienced a multi-region network outage that affected customers across several countries, including the US, UK, Hong Kong, Singapore, Germany, France, Ireland, Spain, and South Africa.

The outage lasted 29 minutes over a 46-minute window, with activity centered around nodes in Newark and London.

Twenty-nine minutes may not sound catastrophic.

But if your payment pipeline, trading workload, SaaS control plane, or cross-region replication stream depended on that path, it was long enough to matter.

And here is the uncomfortable part:

Your cloud provider status page may have stayed green the entire time.

Because the failure may not have been in the cloud service.

It may have been in the network underlay between your users, carriers, and cloud entry points.

This is one of the most underestimated blind spots in cloud resilience.


The Internet is not a single dependency

Recent ThousandEyes reporting shows how noisy and failure-prone the Internet underlay can be.

Across Q1 2026, public reporting based on Cisco ThousandEyes data showed weekly global network outage counts ranging from roughly 199 to 386 events, with the highest week reported between February 23 and March 1.

That does not mean every outage affects your workload.

But it does mean the Internet path your users take to reach your application is constantly changing, degrading, reconverging, and occasionally failing.

And these are not only small ISP issues.

Recent reports highlighted incidents involving major providers such as Arelion, Verizon Business, and Zayo.

The lesson for architects is simple:

Multi-AZ protects you from some infrastructure failures.

Multi-Region protects you from some regional failures.

Neither automatically protects you from every carrier, routing, DNS, last-mile, or underlay reachability failure.


The blind spot: underlay dependency

Most cloud resilience designs stop at the cloud provider boundary.

We design for:

Multi-AZ, Multi-Region, Health checks, Synthetic canaries, Cell-based isolation, Shuffle sharding, Automated failover and so on…

All of that matters.

But it often assumes that the network path between the user and the cloud ingress point is healthy.

That assumption is dangerous.

A carrier-level routing issue can create partial failure modes that are difficult to detect from inside the cloud.

Your endpoint health checks may pass.

Your application metrics may look normal.

Your cloud provider status page may show green.

Meanwhile, customers in a specific geography, ASN, or enterprise network cannot reach you reliably.

That is why “the service is up” and “users can reach the service” are not the same statement.


The DORA and NIS2 angle

For regulated industries, this is not just an engineering concern.

DORA has applied since 17 January 2025 and requires EU financial entities to withstand, respond to, and recover from ICT-related disruptions.

In November 2025, the European Supervisory Authorities designated 19 Critical ICT Third-Party Providers under DORA, including major cloud and technology providers.

The oversight framework is explicitly concerned with systemic and concentration risks created by reliance on a limited number of ICT providers.

That matters because concentration risk is not only a cloud-provider question.

It can also exist in the layers below the cloud:

Carrier paths

DNS dependencies

Internet exchange points

Colocation facilities

Last-mile providers

Shared backbone infrastructure

A multi-cloud design can still share correlated failure domains if the same carrier, exchange, or physical network path appears underneath multiple providers in the same geography.

Resilience analysis needs to go deeper than the provider logo on the architecture diagram.


What architects can do

1. Monitor what you do not own

Use Internet-aware observability.

Amazon CloudWatch Internet Monitor provides visibility into Internet performance and availability between AWS-hosted applications and end users, including issues by client location and network.

External platforms such as ThousandEyes, Catchpoint, and Kentik can provide broader Internet path and BGP-level visibility across providers.

If you cannot observe the underlay, you cannot reason about it.

2. Diversify network entry points

AWS Global Accelerator provides static Anycast IP addresses and brings traffic onto the AWS global network from an edge location close to the user.

For some workloads, that can reduce exposure to long, variable Internet paths between users and regional endpoints.

For hybrid connectivity, design Direct Connect using the appropriate resiliency model.

For critical workloads, AWS describes the Maximum Resiliency model as using separate connections terminating on separate devices in more than one location.

But location diversity is not enough.

You should also validate carrier diversity, physical path diversity, and operational independence.

Two circuits from the same provider, following similar physical paths, may not give you the resilience you think you bought.

3. Build carrier-aware failover

Endpoint health is not enough.

You need to know whether users from key geographies and networks can actually reach the workload.

Use synthetic tests from multiple external vantage points.

Use DNS and traffic steering carefully.

Amazon Application Recovery Controller (ARC) routing controls can help with controlled failover by changing Route 53 health checks through a data-plane-oriented mechanism designed for high availability.

The goal is not only to know whether the endpoint is healthy.

The goal is to know whether the path to the endpoint is healthy for the users who matter.

4. Map the real dependency chain

For critical workloads, document the path from the user to the application.

That includes:

User geography

User network or ASN

DNS resolver behavior

CDN or edge entry point

Carrier path

Cloud ingress

Regional endpoint

Application dependency

For DORA, NIS2, and internal resilience governance, this belongs in the risk conversation.

If the same carrier or facility appears on both the primary and recovery path, you may have a hidden correlated failure domain.


The uncomfortable truth

We have spent years improving resilience inside the cloud.

Multi-AZ.

Multi-Region.

Cell-based architecture.

Chaos engineering.

Automated recovery.

These are real advances.

But Internet and carrier failures remind us that user experience depends on more than the cloud control plane, the application stack, or the database topology.

Resilience does not stop at the cloud provider boundary.

It extends through DNS, routing, carriers, edge networks, enterprise networks, and the last mile.

If your dashboards say “healthy” while your users cannot reach the service, you do not have a cloud availability problem.

You have an end-to-end resilience visibility problem.

The network you do not own can still fail your workload.

Design accordingly.

Get Failover Weekly by email

One essay every two weeks, plus a short note when a new article is published here.

All 25 issues