Have you ever taught a kid to ride a bike? Kids always begin the process by shifting their focus from the handlebars to the pedals, trying to feel out how to keep the right amount of pressure on each pedal, control the handlebars, and keep moving … so they can stay balanced. During this initial learning phase, the kid will keep their eyes down, looking at the pedals, the handlebars, and . . . the ground.
After some time of riding, though, managing the pedals and handlebars are embedded in “muscle memory,” allowing them to get their head up and focus on where they’re going rather than on the mechanical process of riding. After a lot of experience, bike riders can start doing wheelies, or jumps, or off-road riding that goes far beyond basic balance.
Network engineer—any kind of engineering, really—is the same way.
At first, you need to focus on what you are doing. How is this configured? What specific output am I looking for in this show command? What field do I need to use in this data structure to automate that? Where do I look to find out about these fields, defects, etc.?
The problem is—it is easy to get stuck at this level, focusing on configurations, automation, and the “what” of things.
You’re not going to be able to get your head up and think about the longer term—the trail ahead, the end-point you’re trying to reach—until you commit these things to muscle memory.
The point, with technology, is learning to stop focusing on the pedals, the handlebars, and the ground, and start focusing on the goal—whether its nailing this jump or conquering this trail or making it there.
Transitioning is often hard, of course, but its just like riding a bike. You won’t make the transition until you trust your muscle memory a bit at a time.
Learning the theory of how and why things work the way they are is a key point in this transition. Configuration is just the intersection of “how this works” with “what am I trying to do…” If you know how (and why) protocols work, and you know what you’re trying to do, configuration and automation will become a matter of asking the right questions.
Learn the theory, and riding the bike will become second nature—rather than something you must focus on constantly.
I sometimes reference Keith’s Law in my teaching, but I don’t think I’ve ever explained it. Keith’s Law runs something like this:
Any large external step in a system’s capability is the result of many incremental changes within the system.
The reason incremental changes within a system appear as a single large step to outside observers is the smaller changes are normally hidden by abstraction. This is, in fact, the purpose of abstraction—to hide small changes inside a system from external view. Keith’s law is closely related to Clarke’s third law that “Any sufficiently advanced technology is indistinguishable from magic.” What looks like magic from the outside is really just a bunch of smaller things—each easier to understand on its own—combined into one single “thing” through abstraction.
If you’ve read this far, you’re probably thinking—what does this have to do with network engineering?
Well, several things, really.
First—the network is just an abstraction that moves packets to its users. Moving packets seems so … simple … to network users. You put data in here, and data comes out over there. All the little stuff that goes into making a network work are lost in the abstraction of the virtual connection between two hosts.
If you want users to understand why building a network is hard, you’re going to have to work hard at it. And you’re not likely to succeed—it’s often better just to live with the reality that users aren’t going to understand. Of course, this isn’t necessarily a bad thing, at least until it’s time to buy hardware and software to make all this magic work.
Second—no-one outside the network is ever going to understand the refactoring, simplification, and new features you’re trying to build into the network on their own. Users will only understand these things when they are related to some bigger picture, something they can see beyond the abstraction the network presents.
If you’re going to justify doing new things, you need to do so in terms of “larger things,” things that can be seen from outside the abstraction.
Third—no-one is going to pat you on the back for all the little things that need to be done to deploy a new major service. From the outside, that new service, or new cost savings, or whatever—it’s all just indistinguishable from magic.
Keith’s law is both good and bad. But it also means you need to learn how to frame your work in a way that users, who don’t have access to the inner workings of the network, can understand why you’re doing what you’re doing.
Turning this around, this also means you shouldn’t accept the “magic” of vendor products. That brilliant new capability your vendor is showing you is really made up of a lot of smaller components. The abstraction is just that—an abstraction. If you really want to understand the positive and negative consequences of deploying something new, you need to look beyond the abstraction.
We tend to think every technology and every product is roughly unique—so we tend to stay up late at night looking at packet captures and learning how to configure each product individually, and chasing new ones as if they are the brightest new idea (or, in marketing terms, the best thing since sliced bread). Reality check: they aren’t. This applies across life, of course, but especially to technology. From a recent article—
Whenever I start learning a new programming language, I focus on defining variables, writing a statement, and evaluating expressions. Once I have a general understanding of those concepts, I can usually figure out the rest on my own. Most programming languages have some similarities, so once you know one programming language, learning the next one is a matter of figuring out the unique details and recognizing the differences.
RFC1925 rule 11 states—
Rule 11 isn’t just a funny saying—rule 11 is your friend. If want to learn new things quickly, learn rule 11 first. A basic understanding of the theory of networking will carry across all products, all marketing campaigns, and all protocols.
I cannot count the number of times I’ve heard someone ask these two questions—
- What are other people doing?
- What is the best common practice?
While these questions have always bothered me, I could never really put my finger on why. I ran across a journal article recently that helped me understand a bit better. The root of the problem is this—what does best common mean, and how can following the best common produce a set of actions you can be confident will solve your problem?
Bellman and Oorschot say best common practice can mean this is widely implemented. The thinking seems to run something like this: the crowd’s collective wisdom will probably be better than my thinking… more sets of eyes will make for wiser or better decisions. Anyone who has studied the madness of crowds will immediately recognize the folly of this kind of state. Just because a lot of people agree it’s a good idea to jump off a cliff does not mean it is, in fact, a good idea to jump off a cliff.
Perhaps it means something closer to this is no worse than our competitors. If that’s the meaning, though, it’s a pretty cynical result. It’s saying, “I don’t mind condemning myself to mediocrity so long as I see everyone else doing the same thing.” It doesn’t sound like much of a way to grow a business.
The authors do provide their definition—
For a given desired outcome, a “best practice” is a means intended to achieve that outcome, and that is considered to be at least as “good” as the best of other broadly considered means to achieve that same outcome.
The thinking seems to run something like this—it’s likely that everyone else has tried many different ways of doing this; that they have all settled on doing this, this way, means all those other methods are probably not as good as this one for some reason.
Does this work? There’s no way to tell without further investigation. How many of the other folks doing “this” spent serious time trying alternatives, and how many just decided the cheapest way was the best no matter how poor the result might be? In fact, how can we know what the results of doing things “this way” have in all those other networks? Where would we find this kind of information?
In the end, I can’t ever make much sense out of the question, “what is everyone else doing?” Discovering what everyone else is doing might help me eliminate possibilities (that didn’t work for them, so I certainly don’t want to try it), or it might help me understand the positive and negative attributes of a given solution. Still, I don’t understand why “common” should infer “best.”
The best solution for this situation is simply going to be the best solution. Feel free to draw on many sources, but don’t let other people determine what you should be doing.
Those who follow my work know I’ve been focused on building live webinars for the last year or two, but I am still creating pre-recorded material for Pearson. The latest is built from several live webinars which I no longer give; I’ve updated the material and turned them into a seven-hour course called How Networks Really Work. Although I begin here with the “four things,” the focus is on a problem/solution view of routed control planes. From the description:
There are many elements to a networking system, including hosts, virtual hosts, routers, virtual routers, routing protocols, discovery protocols, etc. Each protocol and device (whether virtual or physical) is generally studied as an individual “thing.” It is not common to consider all these parts as components of a system that works together to carry traffic through a network. To show how all these components work together to form a complete system, this video course presents a series of walk throughs showing the processing involved in various kinds of network events, and how control planes use those events to build the information needed to carry traffic through a network.
This course is largely complimentary to the course Ethan and I did a couple of years back, Understanding Network Transports. Taking both would give you a good understanding of network fundamentals. This material is also parallel and complimentary to Problems and Solutions in Computer Networks, which Ethan and I published a few years ago.
I am working on one new live webinar; I really need to get my butt in gear on another one I’ve been discussing for a long time (but I somehow dropped the ball).
I began writing this post just to remind readers this blog does have a number of RSS feeds—but then I thought … well, I probably need to explain why that piece of information is important.
The amount of writing, video, and audio being thrown at the average person today is astounding—so much so that, according to a lot of research, most people in the digital world have resorted to relying on social media as their primary source of news. Why do most people get their news from social media? I’m pretty convinced this is largely a matter of “it saves time.” The resulting feed might not be “perfect,” but it’s “close enough,” and no-one wants to spend time seeking out a wide variety of news sources so they will be better informed.
The problem, in this case, is that “close enough” is really a bad idea. We all tend to live in information bubbles of one form or another (although I’m fully convinced it’s much easier to live in a liberal/progressive bubble, being completely insulated from any news that doesn’t support your worldview, than it is to live in a conservative/traditional one). If you think about the role of social media and the news feed on social media services, this makes some kind of sense. The social media service tries to guess at what will keep you interested (engaged, and therefore coming back to the service), but at the same time each social media service also has a worldview they want to promote. The service largely attempts to both cater to what keeps you there and to pull you towards what the service, itself, believes.
The solution is stop getting your news from social media. period, full stop, end of sentence (although I’ve seen a recent paper indicating people find periods and other punctuation marks offensive in some way—when you find a period offensive, maybe it’s time to grow a little thicker skin).
So how should you get information instead? There are a lot of ways, from email based newsletters to watching television (please don’t, television turns everything into entertainment, including things that are not meant to entertain). My suggestion is, however, is through RSS feeds. Grab an account on Feedly or some other service, find the RSS feeds for the sites you find informative, and subscribe to their feeds. Some services have a learning mechanism that tries to accomplish the same thing as social media feeds—building intelligent filters to emphasize things you find important. I don’t tend to use these things; I have learned to just glance at the headline and first paragraph and make a quick decision about whether I think the post is worth reading.
Following RSS feeds can help you stop binging, jumping from place to place on a single site—essentially wasting time. It works against the mechanisms designers use to “increase engagement,” which often just means to consume more of your attention and time than you intended to give away. Following RSS feeds can also help you gain a broader view of the world if you intentionally subscribe to feeds from sites and people you don’t always agree with. It’s healthy to regularly read “the other side.” Following strong, well-written arguments from “the other side” will do much more for your mind than seeing just the facile, emotionally charged, straw-man arguments often presented (and allowed through the filters) on social media.
Further, services like feedly also allow you to follow lots of other things, including twitter accounts, youtube channels, and podcasts. I follow almost all podcasts through feedly, downloading the individual episodes I want to listen to, storing them in a cloud directory, and then deleting the files when I’m done. This gives me one list of things to listen to, rather than a huge playlist full of seemingly never-ending content.
All this said, this blog has a lot of different RSS feeds available. I don’t have a complete list, but these are a good place to start—
- The main feed (every post other than worth reading): https://rule11.tech/feed/
- Longer written pieces (no podcast, worth reading, posts on other sites, weekend reads, etc.): https://rule11.tech/category/content-type/written/feed/
- The Hedge: https://rule11.tech/category/hedge/feed/
- The History of Networking: https://rule11.tech/category/hon/feed/
I keep these very same links on a page of RSS feeds you can find under the about menu. If you’re interested in the RSS feeds I follow, please reach out to me directly, as feedly no longer has any way to share your feeds other than pushing an OPML file (at least not that I can find).
Decision making, especially in large organizations, fails in many interesting ways. Understanding these failure modes can help us cope with seemingly difficult situations, and learn how to make decisions better. On this episode of the Hedge, Federico Lucifredi, Ethan Banks, and Russ White discuss Federico’s thoughts on developing a taxonomy of indecision. You can find his presentation on this topic here.