When prepend fails, what next? (2)

This week’s post was written by Johnny Britt over at FreedomPay. I’ve edited in some small places to add more information, etc., but I think Johnny needs to start blogging…

Once you have determined that AS-Path prepending can no longer help us what are our next steps? Routing is based on the longest matched prefix, this is true when BGP routes are being compared as well regardless of the AS-PATH. So one option you have is to split your address space into longer advertised prefixes and advertise a slice to each of our upstream providers. In Fig. 1, AS65000 splits its /44 IPv6 into 2 prefixes and advertises them out to AS65001 and AS65004 respectively. This forces half of AS65000 subnet traffic to flow inbound from one specific provider and we can combine both this technique and AS-Path prepending to give us more load sharing capabilities.

Using longer prefixes to direct traffic to a more preferred inbound link can take us a long way in creating the desired inbound traffic pattern. Sometimes there are scenarios where you may need to direct traffic at a more granular level.

But what if you don’t have the ability to create longer prefixes for your ASN (i.e. /24 for IPv4 and a /48 for IPv6)? In these instances, leveraging BGP communities allows us influence how the ASN is advertised within our upstream provider and to its peers. An example can be seen in Fig. 2. AS65002 is a customer of AS65001, as per routing policy within 65001, prefixes learned from customers are preferred over those learned through other BGP peers. This means that in spite of the AS prepend advertised out our link toward AS65001, AS65002 will use the 1 GB link. If AS65002 is a high traffic producer inbound you may be running into a saturated link. Using BGP communities you can tell AS65001 to override its policy and direct traffic toward AS65004 instead.

Most providers have a document someplace that will tell you what communities they accept and use for this sort of task—almost none of them use the communities as listed in RFC1998. For instance, here is an older document from Level 3, and this is JT’s current set of BGP community policies. The best way to find these policies is to ask your provider.

It is possible to have AS prepending, split prefixes, and BGP communities utilized at the same time, but it’s important to be intentional in designing BGP policy. The robustness you gain by applying a more complex policy in BGP could increase fragility in failure scenarios or create more configuration complexity that may be hard to understand at 2 AM. One of the most important questions as engineers we should ask about a design is what happens if something fails. In Fig 3 the combined policies from Fig. 1 and 2 have been applied at the same time. Some questions to think over:

• Should you advertise the aggregate along with our longer prefixes? hint: the answer to this question is almost always yes
• Does AS65004 have a conflicting internal routing policy?
• What happens if you have a link failure?
• Should outbound traffic flow in the same direction in which it was received (stateful firewalling or NAT)?

Nope — this ain’t the last post in this series. Next week I’ll talk about issues with deaggregation, and some other options you can consider in dealing with unequal inbound load on BGP peering.