Simple or Complex?

A few weeks ago, Daniel posted a piece about using different underlay and overlay protocols in a data center fabric. He says:

There is nothing wrong with running BGP in the overlay but I oppose to the argument of it being simpler.

One of the major problems we often face in network engineering—and engineering more broadly—is confusing that which is simple with that which has lower complexity. Simpler things are not always less complex. Let me give you a few examples, all of which are going to be controversial.

When OSPF was first created, it was designed to be a simpler and more efficient form of IS-IS. Instead of using TLVs to encode data, OSPF used fixed-length fields. To process the contents of a TLV, you need to build a case/switch construction where each possible type a separate bit of code. You must count off the correct length for the type of data, or (worse) read a length field and count out where you are in the stream.

Fixed-length fields are just much easier to process. You build a structure matching the layout of the fixed-length fields in memory, then point this structure at the packet contents in-memory. From there, you can just use the structure’s contents to directly access the data.

Over time, however, as new requirements have been pushed into IGPs, OSPF has become much more complex while IS-IS has remained relatively constant (in terms of complexity). IS-IS went through a bit of a mess when transitioning from narrow to wide metrics, but otherwise the IS-IS we use today is the same protocol we used when I first started working on networks (back in the early 1990s).

OSPF’s simplicity, in the end, did not translate into a less complex protocol.

Another example is the way we transport data in BGP. A lot of people do not know that BGP’s original design allowed for carrying information other than straight reachability in the protocol. BGP speakers can negotiate multiple sessions, with each session carrying a different kind of information. Rather than using this mechanism, however, BGP has consistently been extended using address families—because it is simpler to create a new address family than it is to define a new kind of data parallel with address families.

This has resulted in AFs that are all over the place, magic numbers, and all sorts of complexity. The AF solution is simpler, but ultimately more complex.

Returning to Daniel’s example, running a single protocol for underlay and overlay is simpler, while running two different protocols is less simple. However, I’ve observed—many times—that running different protocols for underlay and overlay is less complex.

Why? Daniel mentions a couple of reasons, such as each protocol has a separate purpose, and we’re pushing features into BGP to make it serve the role of an IGP (which is, in the end, going to cause some major outages—if it hasn’t already).

Consider this: is it easier to troubleshoot infrastructure reachability separately from vrf reachability? The answer is obvious—yes! What about security? Is it easier to secure a fabric when the underlay never touches any attached workload? Again—yes!

We get this tradeoff wrong all the time. A lot of times this is because we are afraid of what we do not know. Ten years ago I struggled to convince large operators to run BGP in their networks. Today no-one runs anyone other than BGP—and they all say “but we don’t have anyone who knows OSPF or IS-IS.” I’ve no idea what happened to old-fashioned network engineering. Do people really only have one “protocol slot” in their brains? Can people really only ever learn one protocol?

Or maybe we’ve become so fixated on learning features that we no longer no protocols?

I don’t know the answer to these questions, but I will say this—over the years I’ve learned that simpler is not always less complex.

On the ‘net: Network Models at Packet Pushers

I’ve just started a new series on network models over at Packet Pushers. The first two installments are here:

I learned the Open Systems Interconnect (OSI) model way back in the mid-1980s, as a part of my basic networking education. Ever since then, I’ve used the OSI model in my day-to-day work as a network engineer.

First, models are not sacrosanct. A model is just a tool. If the model you are using is not working for you, feel free to modify it.

On the ‘Net: The IETF at Packet Pushers

I’ve been writing a series about working within the IETF to publish a new standard over at Packet Pushers. The most recent installments are:

There are other seemingly mystical concepts in the IETF process as well—for instance, what is a “document stream,” and what is a document’s “status?”

You’re almost ready to submit a shiny new document to the IETF for consideration, right? Not quite yet—we still need to deal with mandatory sections and language.

Schedule 0923

Here’s a preview of what I’m working on for those who are interested:

  • September 2023: (this Friday) How Routers Really Work, a three-hour live webinar at Safari Books Online through Pearson
  • October 2023:
    • How the Internet Really Works a four-hour live webinar at Safari Books Online through Pearson; this is newly formatted and reorganized version of the two sessions I used to do
    • I’m speaking at a small theological conference on AI and ethics in Cary, NC
  • November 2023:
    • The new CCST book should be released
    • I have recorded a network basics video series that should be released in late 2023 or early 2024
  • January 2024:
    • What Coders Need to Know about Networks, a new course, co-authored with an engineer from Akamai; a three-hour live webinar at Safari Books Online through Pearson
    • I’ll be teaching a course in network engineering at the University of Colorado for the spring semester
  • February 2024: A new three-hour live webinar on infrastructure interviewing skills at Safari Books Online through Pearson
  • March 2024: BGP Policy, a three-hour live webinar on Safari Books Online through Pearson
  • April 2024: Troubleshooting, a reformatted and rebuilt three-hour live webinar at Safari Books Online through Pearson

There will probably be other odds and ends thrown in there, or the schedule might change between now and then. I have courses planned for May and June at Safari Books Online through Pearson, and I’m building two new three-hour courses for July and August of next year.

As for the Hedge, this is what we have coming up:

  • 196: Ethernet Update with Peter J
  • 197: Old Engineering Book Quotes (1)
  • 198: Nephio with Wim Hendricks
  • 199: Clark B on network automation
  • 200: The Internet and civil governance with George Michaelson

We’re always in the process of recording new episodes of the Hedge and looking for new guests and topics—if you would like to come on and talk about something, let me know.

Weekend Reads 091523


The average total cost of a data breach has reached an all-time high in 2023 of $4.45 million. This is an increase of 2.3% from last year’s $4.35 million.


Originally created as a secure sandbox to run compiled C/C++ code in web browsers, WebAssembly (Wasm) has been gaining traction and momentum on the server-side.


Specifically, the web giant’s Privacy Sandbox APIs, a set of ad delivery and analysis technologies, now function in the latest version of the Chrome browser. Website developers can thus write code that calls those APIs to deliver and measure ads to visitors with compatible browsers.


The German digital association, Bitkom, recently announced that the cost of IT equipment theft, data breaches, digital and industrial espionage, and sabotage is expected to reach a staggering 206 billion euros ($224 billion) in 2023.


The alarming rise of phishing attacks has been underscored by a recent study “Phishing Landscape 2023: An Annual Study of the Scope and Distribution of Phishing conducted” by the Interisle Consulting Group, revealing a tripling of such attacks since May 2020.


Japan is widely regarded as one of the most advanced economies for Internet penetration. Japan’s Internet usage rate (individuals) is 82.9% and the development rate of optical fibre is 99.3%.


The robot revolution began long ago, and so did the killing.

Upcoming Training: How Routers Really Work

Have you ever wondered exactly how a router moves a packet from input to output interface? Or what the difference between is between a router’s and host’s operating system? Or why forwarding engines are built in classes, and one forwarding engine cannot “do it all?” Join me on the 22nd at 1pm ET for How Routers Really Work, a three-hour tour through router guts. I’ve replaced about 10% of the slides since the last time I taught this course.

If you register, you can watch the recording at a later date.

Register here.