An I2RS Overview
What is the Interface to the Routing System (I2RS), and why do we need it? To get a good I2RS overview, consider the following illustration for a moment—
What does the interface between, say, BGP and the routing table (RIB) actually look like? What sort of information is carried over this interface, and why? A short (and probably incomplete) list might be—
- Routes being installed by the routing protocol into the RIB—this is the most obvious bit of information, allowing the device to actually build a forwarding table
- Routes being overwritten—this isn’t as obvious as installed routes, but BGP (for instance), can only advertise what is installed in the local table; hence if a route it has installed is overwritten, the process needs to know to stop advertising the route
- Routes being removed from the routing table—perhaps even less obvious, but some routing protocols (BGP is one) allow for multihop routes; when the next hop is removed, any routes using that next hop need to be removed as well
- Connected interfaces removed—this is often handled as a route removal, but it impacts more than just multihop routes; removing a connected route implies loss of reachability to a specific set of neighbors, and hence that set of neighbors need to be moved into a down state, along with recalculation of routing information, etc.
Given this information, what does it look like? Generally speaking, the information traded between routing processes and the RIB flow through a set of Interprocess Communication (IPC) calls within the system itself, and are defined by the code base. For instance, Cisco, Juniper, and OpenSwitch all have fairly unique interfaces between the protocols and the RIB. Cumulus uses Debian UNIX calls for its interface.
Any sort of RIB/protocol interface needs to be very fast. To give a sense of this, consider a quick overview of the install path for a route—
- The protocol runs some internal best/shortest path algorithm based on some event, and determines it needs to install a route
- The protocol makes some call into the RIB to install the route; assume another route already exists for this destination
- As a route is being overwritten, the RIB must call back into the “losing” protocol to notify it
- As a route is being overwritten, the RIB must call back into all the other protocols to validate any routes they’ve installed using this next hop
- Once the route is installed, all the dependent routes must be checked for validity, if they need updates, etc.
- Once the route is installed, the RIB must call into the appropriate “side tables,” such as the label database, neighbor discovery database, and ARP cache, to build layer 2 header rewrite information for the route
- The complete route must be inserted into the FIB—another call into another process, though not using the same API as the RIB/protocol interface
Counting the calls up, it’s easy to see that each installed route can cause six or more passes through the protocol/RIB API, and a couple more in other places. This means that for each route installed in the RIB, the RIB itself must take eight to ten separate actions. In practical terms, this means the RIB must be able to process transactions at a rate around eight to ten times faster than you expect the device to converge. For instance, if the device needs to converge on 250,000 routes in one minute, then the RIB needs to be able to handle on the order of 2 million transactions in that same minute.
Now, let’s assume we wanted to move a routing process off the device, as so—
If we assume protocol x is a process that actually runs on a different device, somewhere out on the network, what would the interface look like? One option is to simply extend the existing proprietary (or open, in some cases) API so it is open to a port number on the box. We could do this by adding another process, a “shim process,” on the box, that listens to the port number and acts as a local “proxy” for off box processes that want to interact with the RIB, as such—
If I wanted to go one step further, I could make the interface between protocol x and the proxy a standard, open, interface. This is precisely what the I2RS specifications do—the proxy is called an agent, and protocol x is called a client. The interface between these two is described using YANG models, which are more commonly used for device configuration.
These models will be examined more closely in a future post on this topic
One immediate question that should spring to mind at this point is—can this remote interface, passing through a standardized model, ever operate as quickly and/or efficiently as a native interface into the device RIB? The clear answer, based on lots of experience and just plain common engineering sense, is no. Then what’s the point of all this work?
The answer to that question will need to wait ’til next time.
Non-time dependent routing policy? 🙂 Blackholing, LFIB manipulation/tunneling, etc. off-box via a standard API? I’ve only just been loosely following the IETF I2RS WG headlines/chatter, so thanks for the high level overview for the lazy.
Steve – i don’t think you’d want to manipulate forwarding over I2RS, it is meant to manipulate control plane, and adheres to basic control plane attributes such as protocol preference/protocol origin/etc. It is up to RIB to select the best route (not necessarily the one provided over I2RS). Data models are rather important there as you could compare intended state (the state after the change has taken place) to the current state and act accordingly.
Jeff — I think it depends on what you consider “manipulating forwarding” — IMHO, the specific use case for I2RS is to manipulate forwarding by injecting RIB entries that override what other routing protocols/manually configured routing information have injected into the RIB. I’m not certain what else you’d use a direct RIB API for than injecting routes? I think we’re probably just using different terminology here.
Russ – I assume to directly manipulate forwarding you’d rather manipulate FIB/LFIB, but you are not and for a right reason, you don’t want to run into race conditions, nor talk to rather difficult to abstract layer, RIB on another side provides enough abstraction to simplify interactions with the system
RIB is not the only producer of forwarding information on the system.
I2RS is there to provide an easy to consume interface rather than pure source of forwarding, think of: next best route, backups of any kind, conditional metadata, etc, not to forget about feedback loop, completely absent in today’s systems
Hey Russ 🙂
I’d also like to emphasize the importance of streaming capabilities as what you are really describing is not a single protocol x but a collection of protocols using single channel to convey bidirectionally and asynchronously information from both sides, and in near real time. (hint – look at gRPC :))
Sure — I’ll get to this at some point. 🙂
Would be great to also discuss a possible consumers of I2RS as well as use of distributed messaging systems