Flowspec and RFC1998?

In a recent comment, Dave Raney asked:

Russ, I read your latest blog post on BGP. I have been curious about another development. Specifically is there still any work related to using BGP Flowspec in a similar fashion to RFC1998. In which a customer of a provider will be able to ask a provider to discard traffic using a flowspec rule at the provider edge. I saw that these were in development and are similar but both appear defunct. BGP Flowspec-ORF https://www.ietf.org/proceedings/93/slides/slides-93-idr-19.pdf BGP Flowspec Redirect https://tools.ietf.org/html/draft-ietf-idr-flowspec-redirect-ip-02.

This is a good question—to which there are two answers. The first is this service does exist. While its not widely publicized, a number of transit providers do, in fact, offer the ability to send them a flowspec community which will cause them to set a filter on their end of the link. This kind of service is immensely useful for countering Distributed Denial of Service (DDoS) attacks, of course. The problem is such services are expensive. The one provider I have personal experience with charges per prefix, and the cost is high enough to make it much less attractive.

Why would the cost be so high? The same reason a lot of providers do not filter for unicast Reverse Path Forwarding (uRPF) failures at scale—per packet filtering is very performance intensive, sometimes requiring recycling the packet in the ASIC. A line card normally able to support x customers without filtering may only be able to support x/2 customers with filtering. The provider has to pay for additional space, power, and configuration (the flowspec rules must be configured and maintained on the customer facing router). All of these things are costs the provider is going to pass on to their customers. The cost is high enough that I know very few people (in fact, so few as to be 0) network operators who will pay for this kind of service.

The second answer is there is another kind of service that is similar to what Dave is asking about. Many DDoS protection services offer their customers the ability to signal a request to the provider to block traffic from a particular source, or to help them manage a DDoS in some other way. This is very similar to the idea of interdomain flowspec, only using a different signaling mechanism. The signaling mechanism, in this case, is designed to allow the provider more leeway in how they respond to the request for help countering the DDoS. This system is called DDoS Open Threats Signaling; you can read more about it at this post I wrote at the ECI Telecom blog. You can also head over to the IETF DOTS WG page, and read through the drafts yourself.

Yes, I do answer reader comments… Sometimes just in email, and sometimes with a post—so comment away, ask questions, etc.

BGP Flowspec Indirection

While Flowspec has been around for a while (RFC5575 was published in 2009), deployment across AS boundaries has been somewhat slow. The primary concern in deploying flowspec is the ability to shoot oneself in the foot, particularly as opening Flowspec to customers can also open apn entirely new, and not well understood, attack surface. Often Flowspec is only offered to customers the provider considers technically competent to reduce this line of risk, or does not offer Flowspec filtering at all.

A second concern is the simple cost of filtering packets. In theory, ASICs can filter packets based on a variety of parameters cheaply. Theory doesn’t always easily translate to practice, however. It is often the case that filtering packets is not a cheap operation, even in the ASIC. The kind and granularity of the filter, and how it is applied, can make a big difference in the cost of implementation.

Regardless, recent work in Flowspec is quite interesting; particularly the ability to redirect flows, rather than simply filtering them. Of course, the original RFCs did allow for the redirection of flows into a VRF on the local router, but this leaves a good bit to be desired. To make such a system work, you must actually have a VRF into which to redirect traffic; for one-off situations, such as directing attack traffic to a honey pot, building the VRF and populating it can be more work than capturing the traffic is worth. A newer draft, draft-ietf-idr-flowspec-path-redirect, aims to resolve this.

Before getting to the draft specifics, however, it is useful to review the basic concept of Flowspec, particularly for readers who might not be familiar with them. Essentially, Flowspec is a new address family carried in BGP where the reachable address (the Network Layer Reachability Information, or NLRI), actually carries a filter set. Communities attached to the NLRI (as attributes) provide a set of actions the router should take if a packet matches the filter criteria carried in the NLRI.

The filter formatting is a bit challenging to understand, as it is set up by bit and nibble, with the ability to string filters together with AND and OR. For instance:

  • You can filter based on a TCP destination port number AND a specific TCP flag by stringing together a type 4 filter and a type 9 filter (in that order specifically), setting the a bit on the second filter in the NLRI to indicate both filter conditions must be matched
  • You can filter based on a range of TCP destination port numbers, say ports 1000-2000, using a type 4 filter with a “greater than or equal to” operator combined with a “1000” value, combined with a “less than or equal to” operator combined with a “2000” value

Just about any combination of port numbers, source addresses, destination addresses, protocol types, and flags can be pulled into a single Flowspec match; hence the difficulty in building ASICs that will support the full range of filtering options in a way that is efficient from a switching perspective and inexpensive (remember: pick two).

In the original Flowspec specification, extended communities are used to describe the action a router should take once a packet has matched a particular filter. For instance—

  • 0x8006: traffic-rate
  • 0x8007: traffic-action
  • 0x8008: redirect
  • 0x8009: traffic-marking

It is 0x8008 that is interesting for the present. This redirect extended community encodes a route target, which in turn identifies a Virtual Routing/Forwarding (VRF) table on the local router. Normally, routes are placed in a particular VRF by attaching a route target community to a route carried in BGP. By matching the target VRF in the Flowspec action with a target VRF in routes carried in BGP, you can dump specific packets into a specific VRF. For instance, if you have two routes to 2001:db8:3e8:100::/64, one of which points to a honeypot box, and the other of which transits your network to a customer’s network, you can—

  • Advertise a route to the same destination as the customer route (2001:db8:3e8:100::/64 in this case) into a special “honeypot” VRF by attaching some community to the route advertisement
  • Detect a problem with traffic sourced from some IP address
  • Advertise the route into the customer’s VRF with a Flowspec rule moving traffic sourced from the attack address and destined to 2001:db8:3e8:100::/64 into the honeypot VRF
  • Resulting in the traffic being forwarded based on the next hop in the honeypot VRF, rather than the customer’s VRF

It would be nice to make the reaction in the example given a bit simpler; this is what draft-ietf-idr-flowspec-path-redirect does. Rather than the “action community” in Flowspec encoding a VRF to redirect traffic in to, the “action community” can point into a redirect action table, or point the packet at a specific tunnel. For instance, if you would like to redirect traffic sourced from address and destined to 2001:db8:3e8:100::/64 into a segment path through your network (ending at the same honeypot, to continue the example), you could—

  • Use the indirection community in the “action community” in the Flowspec advertisement (the actual community string has not been assigned at the time of this writing)
  • Set the indirection id part of the community to Node ID

This directs the traffic to the first node in the segment. Other options are a AS, an SR anycast ID, an SR multicast ID, and several others. The indirection ID can indicate the packet should be redirected according to a local table; in this case, it is up to the operator to figure out how to get the table created and filled with the right information.

This solution is still moderately complex—but BGP is increasingly complex, and Flowspec adds another layer of complexity into the mix. What this solution does do, however, is to allow operators to combine Flowspec with segment routing, or a simple table of operations, that makes it easier to deploy Flowspec in many situations outside the more traditional VPN types of services. This is particularly true for large “enterprise” operators with a BGP core in their network who would like to use Flowspec over a wider range of applications.