We are again here talking about BGP, as promised I will be continuing the conversation diving deeper and deeper in different implementations and uses of the protocol. In this blog post we will explore the pivotal role of network resiliency in ensuring uninterrupted cloud service, focusing on the configuration of your on-prem router (we will be using CISCO as example). We will also discuss how AWS Direct Connect can leverage AS Path Prepending and the strategic implementation of Border Gateway Protocol (BGP) for enhanced reliability.
We will delve into the specifics of AS Path Prepending, a key technique in BGP, which is instrumental in managing traffic flow and ensuring resiliency, especially in environments with multiple Direct Connect circuits. By effectively employing AS Path Prepending, organizations can dictate traffic preferences between primary and secondary Direct Connect links, ensuring seamless service even in the face of potential disruptions. This discussion aims to provide a comprehensive understanding of how to construct a resilient network using CISCO routers and AWS Direct Connect, emphasizing the tactical use of BGP to maintain high availability and robustness in cloud connectivity.
The following topics will be covered in our discussion today:
Introduction to Network Resiliency
In the realm of cloud computing and network architecture, resiliency is the cornerstone of a robust system. Resiliency refers to the ability of a network to maintain optimal performance and uninterrupted service in the face of faults or challenges such as link failures, hardware malfunctions, or heavy traffic loads. The significance of resiliency cannot be overstated; it ensures business continuity, maintains high availability, and provides a seamless user experience. Embracing the principles of the AWS Well-Architected Framework, particularly its emphasis on operational excellence and reliability, is essential in building resilient networks. This framework guides the design and operation of reliable, efficient, and cost-effective systems in the cloud. By integrating resiliency at the core of network design, organizations can safeguard their operations against disruptions, adapt to changes, and emerge resilient in a dynamic digital landscape.
Understanding AS Path Prepending in BGP
AS Path Prepending is a technique used in BGP to influence route selection. BGP, being a path-vector protocol, uses the AS_PATH attribute, among others, to determine the best path to a destination. By artificially inflating the AS_PATH length through prepending, network engineers can make a route less attractive, thereby controlling traffic flow in a multi-homed environment.
The Concept
AS_PATH: A BGP attribute that lists the ASes a route has traversed.
Prepending: Adding extra instances of your own AS number to the AS_PATH.
Impact: Longer AS_PATHs are deemed less desirable, influencing routers to choose shorter paths.
In the context of BGP, a path-vector protocol, the AS_PATH attribute plays a pivotal role in route selection. It records the sequence of Autonomous Systems (ASes) that routing information has traversed. By intentionally lengthening the AS_PATH attribute of a route advertisement via prepending — the process of adding multiple instances of an AS's own number to the AS_PATH — the advertised path appears less preferable compared to shorter alternatives. This approach is particularly effective in multi-homed network scenarios where there are multiple egress or ingress paths. Through AS Path Prepending, network engineers can influence the route choice of upstream or downstream BGP peers, thereby exerting control over traffic direction to manage load balancing, failover, and traffic engineering. The technique is implemented via route-maps or policy-based routing configurations on BGP routers, where specific conditions are defined for selective prepending. This control over path preference is essential for maintaining desired traffic patterns, enhancing network resilience, and optimizing performance in complex network architectures.
Here's an example of what the output might look like when you check the entire BGP routing table on a Cisco router using the show ip bgp command. This output includes several routes, one of which has been modified with AS Path Prepending.
Key Elements in the output above:
BGP table version and router ID: Shows the current version of the BGP table and the router's identifier.
Status and Origin codes: Describes the status (e.g., valid, best) and origin (IGP, EGP, incomplete) of each route.
Network: The IP prefix for the route.
Next Hop: The next-hop IP address to reach the network.
Metric, LocPrf, Weight, Path: Various BGP attributes. Metric (or MED), Local Preference (LocPrf), Weight, and the AS_PATH (Path).
For example, in my lab, the route to 192.0.2.0/24, the path includes multiple instances of AS 64512, indicating AS Path Prepending.
This table provides a comprehensive view of the BGP routing information on the router, including paths that have been manipulated using AS Path Prepending for traffic engineering purposes. The route to 192.0.2.0/24 with the prepended path is less preferred compared to its alternative due to the longer AS_PATH.
So now, let's dive deep and execute the show ip bgp command on a Cisco router, specifically for a route that has been modified using AS Path Prepending, in our lab it's the 192.0.2.0/24.
Key elements in the output above:
BGP routing table entry: Shows the route in question (192.0.2.0/24) and some additional data like version number.
Paths: Indicates the number of paths available for this route and which path is considered the best.
64512 64512 64512 64512 64520: This is the AS_PATH attribute. Here, the AS number 64512 is prepended four times, followed by the originating AS 64520. The prepending makes this route less desirable.
203.0.113.5 from 203.0.113.5 (192.0.2.1): Shows the neighbor IP address that advertised this path and the router interface IP address through which the advertisement was received.
Origin IGP, localpref 100, valid, external: Describes the route's origin, local preference, and other flags like validity and type (external in this case).
Community: Displays any BGP community tags associated with this route.
In this output, you can clearly see how AS Path Prepending affects the route's preference, with the route through AS 64520 being preferred due to a shorter AS_PATH.
AWS Direct Connect and Resiliency
AWS Direct Connect provides dedicated network connections from on-premises to AWS. While it enhances performance and reduces latency, the challenge lies in managing redundancy, particularly for businesses that cannot afford downtime. Implementing AS Path Prepending on these links offers a controlled method for primary and secondary path selection.
Implementing multiple AWS Direct Connect links, as opposed to relying on a single connection, is crucial for ensuring network resiliency in cloud computing. This redundancy is key for maintaining uninterrupted service, as it provides an alternative data pathway in the event of a failure or outage in the primary link. A single Direct Connect, while efficient, poses a risk of a single point of failure, which can lead to significant service disruptions, impacting business operations, customer experience, and potentially leading to revenue loss. By having multiple connections, organizations can ensure high availability and business continuity. This setup not only enables load balancing across connections for optimized performance but also provides the flexibility to implement advanced routing strategies, such as BGP path selection and AS Path Prepending, to control and distribute traffic dynamically between the primary and secondary links, further enhancing the network's robustness and reliability.
Technical Implementation
We will now examine a simple and straightforward architectural model that employs two AWS Direct Connect links. These links are strategically positioned in distinct locations to enhance resiliency. The architecture connects directly to the customer's on-premise edge router. It's important to recognize that while this setup is not the epitome of an optimal solution, it serves a specific educational purpose: to simplify the overall structure, thereby allowing us to concentrate on critical aspects like Border Gateway Protocol (BGP) and Autonomous System (AS) path prepending.
It's crucial to note that in an ideal scenario, our architecture would be more robust. This would be achieved by incorporating multiple on-premise routers and distributing them across various locations. Such an approach significantly elevates the resilience of the network, ensuring continuity and reducing the risk of service disruption.
However, for the scope of this article, we have tailored the architecture to focus primarily on the intricacies of BGP and the strategy of AS path prepending.
Scenario Breakdown
Assume a simple scenario with two AWS Direct Connect links:
Primary Link - High bandwidth, low latency, preferred under normal operations.
Secondary Link - Potentially Lower bandwidth, serves as a backup.
DX Location - Direct Connect Location
Both DX Links - Connecting to DX Gateway.
DX Gateway - Connecting to Transit Gateway.
Transit Gateway - Attached to 2 VPCs, this allow network segmentation.
On-Prem - 2 devices for improved resilience (can be one for dev/test but should be more than one ideally multiple locations).
On-Premises Router Configuration
There are 2 routers in this lab but for the purpose of this lab, we will go through the configuration of one device, the second device will be configured accordingly. Let's consider a Cisco router configuration:
Base Configuration
AS Path Prepending Configuration
In this setup:
AWS_PRIMARY_PEER and AWS_BACKUP_PEER are the IP addresses of the AWS BGP peers for the primary and backup connections, respectively.
The route-map PREPEND_PATH prepends the AS_PATH four times for the backup connection, making it less preferred.
This configuration ensures that under normal circumstances, traffic prefers the primary link.
AWS Side Configuration
I won't spend too much time on the specifics on how to configure the AWS side but I want to take the time on giving some general recommendations.
While AWS’s role in this process is largely passive, it is essential for effectively managing traffic flow and achieving desired routing behaviors. Let's divve deep into the specifics.
AWS's Passive Role in AS Path Prepending:
1. Background: As we discussed above, AS Path Prepending is a BGP technique used to make one path less preferable than others. It involves appending multiple instances of your own ASN (Autonomous System Number) to the AS_PATH attribute in BGP route announcements. This makes the path appear longer and thus less desirable to BGP route decision processes.
2. AWS’s Passive Nature: AWS does not actively prepend AS paths in your Direct Connect configurations. Instead, it passively accepts the AS_PATH attribute that your on-premises network advertises. The length of the AS_PATH influences AWS’s BGP route selection process.
Key Considerations for Effective Configuration:
1. Configuring AWS VPC Route Tables:
- Ensure your VPC route tables are set up to recognize and properly route traffic based on the paths influenced by AS Path Prepending.
- This involves configuring route tables to accommodate routes with varying AS_PATH lengths, allowing you to control which path is preferred for ingress traffic into your VPC.
2. Use of BGP Communities (more on this on future BGP blog posts/ideal for multiple DX location deployments):
- BGP communities are optional tags that can be attached to route announcements. They can be used to apply routing policies or signal specific handling of routes. This blog post put focus on AS-Path Prepending.
- If you use BGP communities, ensure they are correctly set up to influence routing decisions on the AWS side.
- AWS may use these community tags to modify the treatment of routes (such as preferring one route over another, depending on the confuguration this can affect the route manipulation based on the BGP Best Path Algorithm, more on future blog posts).
3. AWS-side Route Selection:
- Assuming there are not additional variables with higher priority, AWS's route selection will favor shorter AS paths. By increasing the length of the AS_PATH attribute via prepending, you can deprioritize certain routes.
- This is particularly useful in scenarios like ours, where you have multiple Direct Connect links and want to control primary and secondary path selections.
Practical Outcome:
In a scenario with primary and secondary Direct Connect links, you might prepend the AS_PATH for routes advertised over the secondary link. This would make the primary link more favorable for incoming traffic to your AWS resources. However, should the primary link fail, the secondary link with the longer AS_PATH would still provide a viable alternative route.
Remember, while AWS’s role in this process is more about accepting and acting upon the AS_PATH lengths you configure on your on-prem router, the effectiveness of AS Path Prepending depends largely on your configuration and the design of your network architecture.
Monitoring and Validation
BGP Route Analysis: Regularly analyze the BGP routes in both AWS and your on-premises routers to confirm the prepend is effective. You can do this with the show ip commands we used above here. You can also use traceroute as simple tool to verify the traffic direction.
Failover Simulation: Conduct controlled failovers to ensure the traffic shifts as expected.
Performance Metrics: Track metrics like latency, jitter, and packet loss during normal operation and failovers.
Advanced Considerations
Dynamic Prepending: Consider scripts or automation tools that adjust prepend length based on real-time network performance metrics.
This is an example of a simple script to monitor network performance metrics like latency, packet loss, or throughput. Based on the metrics, the script can adjust the prepend lenght.
The script would run on a separate server or device, continuously monitoring network performance and updating the router configuration via SSH as needed.
Please Note:
Customization: The script needs to be customized for the specific network environment and performance criteria.
Security: Ensure secure methods (like SSH with keys) for the script to interact with the router.
Resilience: The script should have error handling and be resilient to network issues.
Testing: Thoroughly test the script in a lab environment before deploying in a production network.
Selective Prepending: Apply prepending to specific routes rather than all routes, based on business needs.
Prefix-List (PL-PREPEND-SEQ10): Adjust the prefix 192.168.100.0/24 to match the specific route(s) you wish to prepend.
AS Path Prepend: Change 65001 to your actual ASN. The number of times it is repeated (65001 65001 65001 65001) determines the prepend length.
BGP Neighbor Configuration: Replace 203.0.113.1 and 65002 with your actual BGP neighbor's IP address and ASN (this is just an example taken from my lab).
Testing: Always remember, it's important to test this configuration in a controlled environment before deploying it in a live network to ensure it behaves as expected.
Monitoring: After implementation, keep an eye on the BGP route advertisements and network traffic to ensure the prepending is achieving the desired effect.
In providing the aforementioned examples of Cisco router configurations, it's important to emphasize that these were formulated and tested within a controlled lab environment.
However, it is crucial to understand that these configurations serve primarily as illustrative examples. They are intended to demonstrate the principles and techniques of BGP AS path prepending in a clear and tangible manner. As such, they are not ready-made solutions for immediate implementation in a production environment.
Each production environment is unique, with its own set of challenges, requirements, and constraints. Therefore, these example configurations must be thoughtfully analyzed and thoroughly customized to align with the specific needs and characteristics of your production environment. This customization process should take into account factors such as your network topology, traffic patterns, security policies, and the specific objectives you aim to achieve through these configurations.
I strongly recommend conducting a comprehensive review and adaptation of these configurations, followed by thorough testing in a staging environment that closely mirrors your production network. This approach will help ensure that the configurations are not only technically sound but also perfectly tailored to support the operational goals and requirements of your network infrastructure.
Conclusion
AS Path Prepending is a powerful yet nuanced tool in BGP for controlling traffic paths. Its implementation in AWS Direct Connect environments must be meticulously planned, executed, and continually monitored. The blend of AWS's cloud infrastructure with robust on-premises BGP configurations paves the way for resilient, fail-safe network architectures essential for modern businesses.
Remember..... at some point, no matter how resilient is the service you are using, it will fail, and when it does, you need to be prepared!
Thanks for taking the time to read, see you soon!