About that Easy Button …
We love layers and abstraction. After all, building in layers and it’s corollary, abstraction, are the foundation of large-scale system design. The only way to build large-scale systems is to divide and conquer, which means building many different component parts with clear and defined interaction surfaces (most often expressed as APIs) and combining these many different parts into a complete system. But abstraction, layering, and modularization have negative aspects as well as positive ones. For instance, according to the State/Optimization/Surface triad, any time we remove state in order to control complexity, we either add an interaction surface (which adds complexity) or we reduce optimization.
Another impact of abstraction, though, is the side effect of Conway’s Law: “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.” The structure of the organization that designs a system is ultimately baked into the modularization, abstraction, and API schemes of the system itself.
To take a networking instance, many networks use one kind of module for data centers and another for campuses. The style of network built in each place, where the lines are between these different topological locations in the network, the kind of policies expressed at the borders of these networks, and many other factors are baked into the network design based on the organizational design. This is all well and good within an organization, but what happens when you reach outside the organization and purchase hardware and software from another company to build your network?
When you buy systems designed by other organizations, you are importing their organizational structure into your organization. A corollary: the more vertically integrated a system is, or more aggregated, the more of the external organization’s structure you are importing into your organization. It seems like purchasing a strongly integrated system is “pressing the easy button,” but the result is often actually a mess of tech debt from trying to make the organizing principles of the purchased system fit with the internal logic of your organization.
This is why, for instance, some software developers advocate not using open source libraries and frameworks in building internal projects. The idea sounds radical, contrary to the direction of the entire tech culture. The easy button calls; the network is just an undifferentiated bit moving machine, just give me a black box and two lines of configuration, and I’m happy. I don’t want to know how it works.
Until it doesn’t. Or until it doesn’t do what you want for the thousandth time, and you add the thousandth line of configuration to tweak something, or you run into the thousandth assumption in the way the black box works that you just need to change to make your business run better. When you look at the system architecture as a reflection of organizational structure, these little mismatches begin to make sense.
Lessons for the network? It isn’t really possible to build all the bits and pieces you need to build a network. You are not going to be able to write your own network operating system. You are not going to be able to (from scratch) build your own control plane.
But you can be more careful about the organizational structures you are importing. You can understand the internal components and how they connect. You can understand the problems being solved, the general solution being implemented, and how these solutions ultimately fit together to make a whole.
You can insist on having access to the same APIs the system developers use. You can minimize the number of system elements you rely on, such as protocols and nerd knobs. You can, ultimately, disaggregate, treating your software and hardware as two different “things,” each of which has its own lifecycle, purposes, and value.
This is as much about mindset as anything else; the easy button is an abstraction. Abstractions are abstracting something. Don’t just watch the vendor presentation and gawk; dig in and understand.