CONTENT TYPE
The Hedge 69: Container Networking Done Right
Everyone who’s heard me talk about container networking knows I think it’s a bit of a disaster. This is what you get, though, when someone says “that’s really complex, I can discard the years of experience others have in designing this sort of thing and build something a lot simpler…” The result is usually something that’s more complex. Alex Pollitt joins Tom Ammon and I to discuss container networking, and new options that do container networking right.
Rethinking BGP on the DC Fabric
Everyone uses BGP for DC underlays now because … well, just because everyone does. After all, there’s an RFC explaining the idea, every tool in the world supports BGP for the underlay, and every vendor out there recommends some form of BGP in their design documents.
I’m going to swim against the current for the moment and spend a couple of weeks here discussing the case against BGP as a DC underlay protocol. I’m not the only one swimming against this particular current, of course—there are at least three proposals in the IETF (more, if you count things that will probably never be deployed) proposing link-state alternatives to BGP. If BGP is so ideal for DC fabric underlays, then why are so many smart people (at least they seem to be smart) working on finding another solution?
But before I get into my reasoning, it’s probably best to define a few things.
In a properly design data center, there are at least three control planes. The first of these I’ll call the application overlay. This control plane generally runs host-to-host, providing routing between applications, containers, or virtual machines. Kubernetes networking would be an example of an application overlay control plane.
The second of these I’ll call the infrastructure overlay. This is generally going to be eVPN running BGP, most likely with VXLAN encapsulation, and potentially with segment routing for traffic steering support. This control plane will typically run on either workload supporting hosts, providing routing for the hypervisor or internal bridge, or on the Top of Rack (ToR) routers (switches, but who knows what “router” and “switch” even mean any longer?).
Now notice that not all networks will have both application and infrastructure overlays—many data center fabrics will have one or the other. It’s okay for a data center fabric to only have one of these two overlays—whether one or both are needed is really a matter of local application and business requirements. I also expect both of these to use either BGP or some form of controller-based control plane. BGP was originally designed to be an overlay control plane; it only makes sense to use it where an overlay is required.
I’ll call the third control plane the infrastructure underlay. This control plane provides reachability for the tunnel head- and tail-ends. Plain IPv4 or IPv6 transport is supported here; perhaps some might inject MPLS as well.
My argument, over the next couple of weeks, is BGP is not the best possible choice for the infrastructure underlay. What I’m not arguing is every network that runs BGP as the infrastructure underlay needs to be ripped out and replaced, or that BGP is an awful, horrible, no-good choice. I’m arguing there are very good reasons not to use BGP for the infrastructure underlay—that we need to start reconsidering our monolithic assumption that BGP is the “only” or “best” choice.
I’m out of words for this week; I’ll begin the argument proper in my next post… stay tuned.
The Hedge 66: Daniel Migault and the ADD Working Group
The modern DNS landscape is becoming complex even for the end user. With the advent of so many public resolvers, DNS over TLS (DoT) and DNS over HTTPS (DoH), choosing a DNS resolver has become an important task. The ADD working group will, according to their page—
In this episode of the Hedge, Daniel Migault joins Alvaro Retana and Russ White to discuss Requirements for Discovering Designated Resolvers, draft-box-add-requirements-02.
Agglutinating Problems Considered Harmful (RFC2915, Rule 5)
In the networking world, many equate simplicity with the fewest number of moving parts. According to this line of thinking, if there are 100 routers, 10 firewalls, 3 control planes, and 4 management systems in a network, then reducing the number of routers to 95, the number of firewalls to 8, the number of control planes to 1, and the number of management systems to 3 would make the system “much simpler.” Disregarding the reduction in the number of management systems, scientifically proven to always increase in number, it does seem that reducing the number of physical devices, protocols in use, etc., would tend to decrease the complexity of the network.
The wise engineers of the IETF, however, has a word of warning in this area that all network engineers should heed. According to RFC1925, rule 5: “It is always possible to agglutinate multiple separate problems into a single complex interdependent solution. In most cases this is a bad idea.” When “conventional wisdom” and the wisdom of engineers with the kind of experience and background as those who write IETF documents contradict one another, it is worth taking a deeper look.
A good place to begin is with other RFCs that might provide examples, or otherwise shed light on this situation. Two of particular interest are RFC1776 and RFC3093.
RFC1776 describes a very simplified transport protocol for use in the Internet and private networks. In normal packet formats there are many different components, such as a header and data sections. The header is normally made up of many different fields, such as the source address, the destination address, the quality of service, etc. The data section of the packet may also be divided into many different fields providing information for such functionality as error detection, flow control, and indicators of which application on the destination host this information is destined to (the port number is an example).
The authors of RFC1776 decided that the wisdom of making a single appliance which provides many services, the firewall being the classic example, and the wisdom of using a single protocol for everything, for instance using BGP for data center fabrics and interdomain connectivity, should be applied fully to the formatting of transport packets. In the spirit of agglutination common to all network engineering, RFC1776 recommends replacing the entire contents of a transport packet with a single address. The address must be a bit longer, of course, to carry the actual data, but using a single large field is inherently simpler than using many different fields. To accomplish this task, RFC1776 specifies a packet with 1696 octets (bytes) of address space. The number of octets originally selected is compatible with ATM, an older technology which uses a 53-octet cell but should also be compatible with all modern transport systems.
While the many advantages of this system are not fully described in the specification, it should be obvious packets containing a single field—the destination address—will be easier to hosts to generate and transmit, and easier for hosts to receive and process. The entire processing of the packet will just be transferring the address field directly into memory for consumption by any application running on the host that desires to consume it. The specification does note, however, that security is much simpler because there is no “user data” to secure.
RFC3093, a more recent example of agglutination in order to simplify network design and operation. This authors of this RFC note that applications are already moving to using a single port, 80, for all traffic, as most firewalls already pass traffic transmitted through this port without restrictions. The authors note the operation of the Internet would be much simpler if all applications ran over port 80. In this way, all applications could pass through firewalls without modification, while the firewalls themselves remain perfectly operational, fulfilling their intended purpose. Implementing this specification would also simplify the absolute mess of port and protocol numbers used in transporting data today, agglutinating them all down to a single port. As less is always simpler, this would create a simpler, easier to manage, global Internet.
The lessons to learn, after examining the options, may not be what was originally intended. Reducing the number of parts does not necessarily reduce the complexity of the overall system. If you haven’t found the tradeoffs, you haven’t looked hard enough.
Focus is a Virtue
The modern world craves our attention—but only in short bursts. To give your attention to any one thing for too long is failing, it seems, because you might miss out on something else of interest. We have entered the long tail of the attention economy, grounded in finding every smaller slices of time in which the user’s attention can be captured and used.
The problem is obvious for anyone with eyes to see. What is the solution? The good news is there are solutions. The bad news is these solutions are swimming upstream against the major commercial interests of our day, so it’s going to take work and determination. The problems are platform based, while the solutions are personal and hard.
Begin here—treat focus as a virtue. We normally think of virtues as things like being kind, or giving money to charity, or (in the modern world) signaling that we support the right things and think the right way. But virtue reaches far deeper than just believing the right things or being “nice.”
Sertillanges, in The Intellectual Life, says the virtues are bound together. A person with only one virtue will often find that virtue twisted into something it is not. A clump of trees, no matter how small, is more likely to survive a storm than the singular tree, no matter how strong the single tree might be. You must not only develop the virtue of quick thinking, or of curiosity, but also of focus. \
How do you develop focus? I can tell you the wrong way: try to make yourself focus for a long period of time. Maybe this will work for some people but forcing attention onto a single topic often backfires in very spectacular ways.
Instead, I would counsel a two-step program: eliminate distractions and expand slowly.
Sertillanges says, “As to the public, if it sometimes stimulates, it often disturbs, scatters the mind; and by going to pick up two pennies in the street, you may lose a fortune.” What is social media other than “the public?” Simply having too much information to hand can also be problematic, as well—”There are books everywhere and only a few are necessary.”
A distraction people don’t often think about is reading too many books at once. Most of the people I know are reading (if they are, really) five or ten books at once. They switch back and forth between books, picking up a little here and a little there. It’s a dandy application of multitasking to an old technology.
But I don’t think it actually works. Pick up a book and read it. Learn to follow the line of a single argument from start to finish. I find it helpful to outline information-rich books, or books that have complex lines of argument. The act of rethinking what the author is saying, and rebuilding their line of thought, is really helpful.
As for expanding slowly, this means two things. First, don’t try to jump from a six-minute attention span to an hour-long attention span in a day. Try to go from six minutes to eight, and then eight to twelve, etc. Don’t try to have an infinite attention span, either—it just is not humanly possible. Setting unrealistic goals is a recipe for failure.
Second, expand slowly by building mental maps, rather than trying to consume the outer shell first. The outer shell (“what does this command do?”) might be the most immediately useful, but if you stay on the outer shell your entire life, jumping all over the place to find the next bit of useful information, you’re never going to learn to focus.
Further, if you jump all over the place, you’re never going to build the mental maps that will allow you to focus. When I first started reading philosophy, I was often more confused than anything else. It was like jumping into the middle of a conversation—there were (and still are) terms and ideas I had no idea how to relate to. Over time, I built a mental map. While I’m still not able to read philosophy (or theology) like many of my friends who have spent their lives reading this stuff, I am at least becoming somewhat competent.
So… slow down. Remove distractions. Set goals. Build mental maps.
If you want to find the path to success in life, it is going to be through focus.
The Hedge 67: Daniel Beveridge and the Structure of Innovation
Innovation and disruption are part the air we breath in the information technology world. But what is innovation, and how do we become innovators? When you see someone who has invented a lot of things, either shown in patents or standards or software, you might wonder how you can become an innovator, too. In this episode of the Hedge, Tom Ammon, Eyvonne Sharp, and Russ White talk to Daniel Beveridge about the structure of innovation—how to position yourself in a place where you can innovate, and how to launch innovation.
IPv6 Buzz: Is IPv6 Baked Enough?
I was recently a guest on the IPv6 Buzz podcast. Ed, Scott, Tom, and I talk about IPv6 operational maturity, IPv6 standards, and the IETF process. This was a great episode, you should really listen to it … and listen to IPv6 Buzz in general.