DC Fabric Segment Routing Use Case (4)
In the last post in this series, I discussed using SR labels to direct traffic from one flow onto, and from other flows off of, a particular path through a DC fabric. Throughout this series, though, I’ve been using node (or prefix) SIDs to direct the traffic. There is another kind of SID in SR that needs to be considered—the adj-sid. Let’s consider the same fabric used throughout this series—
So far, I’ve been describing the green marked path using the node or (loopback) prefix-sids:
[A,F,G,D,E]. What’s interesting is I could describe the same path using adj-sids:
[a->f,f->g,g->d,d->e], where the vector in each hop is described by a single entry on the SR stack. There is, in fact, no difference between the two ways of describing this path, as there is only one link between each pair of routers in the path. This means everything discussed in this series so far could be accomplished with either a set of adj SIDs ore a set of node (prefix) SIDs.
Given this, why are both types of SIDs defined? Assume we zoom in a little on the border leaf node in this topology, and find—
- Router E, a ToR on the “other side of the fabric,” wants to send traffic through Y rather than Z
- Router Y does not support SR, or is not connected to the SR control plane running on the fabric
What can Router E do to make certain traffic exits the border leaf towards Router Y, rather than Router Z? Begin with Router A building an adj-sid for each of it’s connections to the exiting routers—
- A->Y on A1 is given label 1500
- A->Y on A2 is given label 1501
- A->Y ECMP across both links is given label 1502
- A->Z is given label 1503
With Router advertising this set of labels, E can use the SR stack—
[H,A,1500]to push traffic towards Y along the A1 link
[H,A,1501]to push traffic towards Y along the A2 link
[H,A,1502]to push traffic towards Y using ECMP using both A1 and A2
This indicates there are two specific places where adj-sids can be useful in combination with node (or prefix) SIDs—
- When the router towards which traffic needs to be directed doesn’t participate in SR; this effectively widens the scope of SR’s effectiveness to one hop beyond the SR deployment itself
- When there is more than one link between to adjacent routers; this allows SR to choose one of a set of links among a group of bundled links, or a pair of parallel links that would normally be used through ECMP
Again, note that the entire deployment developed so far in this series could use either just adj-sids, or just node (prefix) SIDs; both are not required. To gain more finely grained control over the path taken through the fabric, and in situations where SR wants to be extended beyond the reach of the SR deployment itself, both types of SIDs are required.
This brings up another potential use case in a data center fabric, as well. So far, we’ve not dealt with overlay networks, only the fabric itself. Suppose, however, you have—
Where C, D, E, and F are ToR or leaf switches (routers), and A, B, G, and H are hosts (or switches inside hypervisors). Each of the two differently colored/dashed sets of lines represent a security domain of some type; however, this security domain appears, from IP’s perspective, to be a single subnet. This could be, for instance, a single subnet split into multiple broadcast domains using eVPNs, or simply some set of segments set up for security purposes. In this particular situation, G wants to send traffic to D without F receiving it. Given the servers themselves participate in the SR domain, how could this be accomplished? G could send this traffic with an SR stack telling E to transmit the traffic through the blue interface, rather than the green one. This could be a single SR stack, containing just a single adj-sid telling E to which interface to use when forwarding this traffic.
I had hoped to finish this series this week, but it looks like it’s going to take one more week to wrap up a few odds and ends…
some comments,there are more details in the SR webcast I did with Ivan.
1. adj-sids in most implementations are local (flag L set) and as such only meaningful to the originating node, moreover, some implementations don’t support global adj-sids at all. This means that an adj-sid must be programmed in context of a nodal-sid
2. assuming global adj-sid is supported forming an end2end “strict” path is problematic from label stack depth prospective (HW support) since it becomes a function of number of links, where every link is represented by a label in the stack pushed at ingress. If global adj-sid is not supported the issue is even more severe since now every hop in the path would be represented by 2 labels (nodal-sid,adj-sid)
So from practical prospective a path computation entity (PCE :)) should only use adj-sid in cases where nodal-sid for the next hop doesn’t provide a non-ambiguous path, which should result in a mix of loose and strict segments
Thanks, Jeff — that’s the point I was trying to get to — that the adj-sid should really only be used in the one specific situation, when the node/prefix sid doesn’t provide an unambiguous path through the network. I wanted to get to it from the “why” angle, rather than just stating it, though. 🙂 The label stack depth problem just occurred to me — in reality, with most hardware, you wouldn’t even be able to describe the full path across a 5 stage spine and leaf of the kind I’ve been using as an example. I might do “yet another post” on limitations of this sort, if I find enough of them to discuss.
Russ, current limitations are:
merchant low end, 4 labels or 4+1 (transport + service)
merchant high end 8-14, vendor high end 10+, I recall Orange did some testing presented at MPLS/SDN WC last year or year before, if I find it, I’ll provide more info.
From control plane prospective MSD (Max SID depth) could be signaled thru extensions defined:
Typo => “G could send this traffic with an SR stack telling E to transmit the traffic through the blue interface, rather than the blue one.” It should be “G could send this traffic with an SR stack telling E to transmit the traffic through the blue interface, rather than the GREEN one.”
The main advantage of using adjacency sids is to use sub-optimal/under utilized paths for traffic which is less delay sensitive.
Thanks for catching this! 🙂 Russ