WRITTEN
Focus is a Virtue
The modern world craves our attention—but only in short bursts. To give your attention to any one thing for too long is failing, it seems, because you might miss out on something else of interest. We have entered the long tail of the attention economy, grounded in finding every smaller slices of time in which the user’s attention can be captured and used.
The problem is obvious for anyone with eyes to see. What is the solution? The good news is there are solutions. The bad news is these solutions are swimming upstream against the major commercial interests of our day, so it’s going to take work and determination. The problems are platform based, while the solutions are personal and hard.
Begin here—treat focus as a virtue. We normally think of virtues as things like being kind, or giving money to charity, or (in the modern world) signaling that we support the right things and think the right way. But virtue reaches far deeper than just believing the right things or being “nice.”
Sertillanges, in The Intellectual Life, says the virtues are bound together. A person with only one virtue will often find that virtue twisted into something it is not. A clump of trees, no matter how small, is more likely to survive a storm than the singular tree, no matter how strong the single tree might be. You must not only develop the virtue of quick thinking, or of curiosity, but also of focus. \
How do you develop focus? I can tell you the wrong way: try to make yourself focus for a long period of time. Maybe this will work for some people but forcing attention onto a single topic often backfires in very spectacular ways.
Instead, I would counsel a two-step program: eliminate distractions and expand slowly.
Sertillanges says, “As to the public, if it sometimes stimulates, it often disturbs, scatters the mind; and by going to pick up two pennies in the street, you may lose a fortune.” What is social media other than “the public?” Simply having too much information to hand can also be problematic, as well—”There are books everywhere and only a few are necessary.”
A distraction people don’t often think about is reading too many books at once. Most of the people I know are reading (if they are, really) five or ten books at once. They switch back and forth between books, picking up a little here and a little there. It’s a dandy application of multitasking to an old technology.
But I don’t think it actually works. Pick up a book and read it. Learn to follow the line of a single argument from start to finish. I find it helpful to outline information-rich books, or books that have complex lines of argument. The act of rethinking what the author is saying, and rebuilding their line of thought, is really helpful.
As for expanding slowly, this means two things. First, don’t try to jump from a six-minute attention span to an hour-long attention span in a day. Try to go from six minutes to eight, and then eight to twelve, etc. Don’t try to have an infinite attention span, either—it just is not humanly possible. Setting unrealistic goals is a recipe for failure.
Second, expand slowly by building mental maps, rather than trying to consume the outer shell first. The outer shell (“what does this command do?”) might be the most immediately useful, but if you stay on the outer shell your entire life, jumping all over the place to find the next bit of useful information, you’re never going to learn to focus.
Further, if you jump all over the place, you’re never going to build the mental maps that will allow you to focus. When I first started reading philosophy, I was often more confused than anything else. It was like jumping into the middle of a conversation—there were (and still are) terms and ideas I had no idea how to relate to. Over time, I built a mental map. While I’m still not able to read philosophy (or theology) like many of my friends who have spent their lives reading this stuff, I am at least becoming somewhat competent.
So… slow down. Remove distractions. Set goals. Build mental maps.
If you want to find the path to success in life, it is going to be through focus.
Technologies that Didn’t: ARCnet
In the late 1980’s, I worked at a small value added reseller (VAR) around New York City. While we deployed a lot of thinnet (RG58 coax based Ethernet for those who don’t know what thinnet is), we also had multiple customers who used ARCnet.
Back in the early days of personal computers like the Amiga 500, the 8086 based XT (running at 4.77MHz), and the 8088 based AT, all networks were effectively wide area, used to connect PDP-11’s and similar gear between college campuses and research institutions. ARCnet was developed in 1976, and became popular in the early 1980’s, because it was, at that point, the only available local area networking solution for personal computers.
ARCnet was not an accidental choice in the networks I supported at the time. While thinnet was widely available, it required running coax cable. The only twisted pair Ethernet standard available at the time required new cables to be run through buildings, which could often be an expensive proposition. For instance, one of the places that relied heavily on ARCnet was a legal office in a small town in north-central New Jersey. This law office had started out in an older home over a shop in the square of a smaller town—a truly historic building well over a hundred years old. As the law office grew, they purchased adjacency buildings, and created connecting corridors through closets and existing halls by carefully opening up passages between the buildings. The basements of the buildings were more-or-less connected anyway, so the original telephone cabling was tied together to create a unified system.
When the law office decided to bring email and shared printers up on Novell Netware, they called in the VAR I worked for to figure out how to make it all work. The problem we encountered was the building had been insulated at some point with asbestos fiber filling in the walls. Wiring on the surface of the walls and baseboards was rejected because it would destroy the historical character of the building. Running through the walls would only be possible if the asbestos was torn out—this would be removing the walls, again encountering major problems with the historical nature of the building.
The solution? ARCnet can run on the wiring used for plain old telephone circuits. Not very fast, of course; the original specification was 2.5Mbit/s. On the other hand, it was fast enough for printers and email before the days of huge image files and cute cat videos. ARCnet could also run in a “star” configuration, which means with a centralized hub (which we would today call a switch), and each host attached as a spoke or point on the star. This kind of wiring had just been introduced for Ethernet, and so was considered novel, but not widely deployed.
ARCnet deployed to well over ten thousand networks globally (a lot of networks for that time period), and then was rapidly replaced by Ethernet. The official reason for this rapid replacement was the greater speed of Ethernet—but as I noted above, most of the applications for networks in those days did not really make use of all that bandwidth, even in larger networks. Routers were not a “thing” at this time, but you could still connect several hundred hosts onto a single ARCnet or Ethernet segment and expect it to work with the common traffic requirements of the day.
At the small VAR I worked at, we had another reason for replacing ARCnet: it blew up too much. The cables over which POTs services run is unshielded, and hence liable to induced high voltage spikes from other sources. For instance, we had to be quite intentional about not using a POTs lines located within a certain distance of the older wiring in the buildings where it was deployed; a voltage spike could not only cause the network to “blank out” for some amount of time, it could actually cause enough voltage on the wires to destroy the network interface cards. We purchased ARCnet interface cards by the case, it seemed. After any heavy thunderstorm, the entire shop went from one ARCnet customer to another replacing cards. At some point, replacing cases of interface cards becomes more expensive than performing asbestos mitigation, or even just running the shielded cable Ethernet on twisted pair requires. It becomes cheaper to replace ARCnet than it does to keep it running.
An interesting twist to this story—there is current work in the Ethernet working group of the IEEE to make Ethernet run on … the cabling used for very old POTs services. This is effectively the same use case ARCnet filled for many VARs in the late 1980’s. The difference, today, is that much more is understood about how to build electronics that can support high voltage spikes while still being able to discriminate a signal on a poor transmission medium. Much of this work has been done for the wireless world already.
So ARCnet failed more because it was a technology ahead of its time, in terms of its use case, but in line with its time, in its physical and electronic design.
The OSI Model: STOP IT!
The OSI model is perhaps the best-known—and perhaps the most-loved—model in the networking world. It’s taught in every basic networking course, and just about every blog (other than this one) has some article explaining the model someplace or another (for instance, here is one of the better examples).
The reality is, however, that I’ve been in the networking business for 30’ish years and I’ve never once used the OSI model for anything practical. I’ve used the model when writing books because just about every book on networking has to have a section on the OSI model. I’ve used the model when writing a paper comparing two different protocols, back in the multiprotocol days (VIP versus IPX versus IP), but we don’t have those kinds of arguments very often any longer.
So, we all learn the OSI model, and yet I don’t know of anyone who actually uses the OSI model in understanding how protocols work, or how to troubleshoot a network. There’s the “it’s a layer two problem” statement, and that’s about the end its useful life, it seems.
Let me make a suggestion—learn, use, and teach the RINA model instead. Okay, so what is the RINA model? It is the Recursive Internet Network Architecture, hence RINA. The observation John Day made when creating the RINA model is this: there are only four problems you need to solve to move data from one host to another. It just happens that those four problems need to be solved multiple times: across each physical link, between each pair of hosts, and then again between each pair of applications (at least—John Day argues there should be eight layers rather than seven, but that argument is outside the scope of this simple blog post).
The four problems are: transport (which I would call marshaling), multiplexing, error control, and flow control. Protocols that solve these problems tend to solve two of the four problems rather than one; the pairings almost always seem to be marshaling with multiplexing and error control with flow control.
On a single Ethernet hop, multiplexing and marshaling are generally solved by the physical link Ethernet protocol. Error and flow control, on the other hand, are generally solved by the data link protocol (something we don’t much think about any longer). Again, multiplexing and marshaling are generally solved by IP, while error and flow control are generally solved by TCP.
The model doesn’t perfectly describe the purpose of every protocol in the modern networking stack, but it comes much closer than the OSI model. Since it’s recursive, you can even insert layers to represent tunnels without a lot of hand-waving in the process, or “breaking” the model.
Why is the RINA model better for understanding protocols? Because it describes what the protocol is trying to accomplish, rather than where it lives in the stack, or what kinds of interfaces it provides. Why is the RINA model better for troubleshooting? Because if you know what isn’t working, it gives you a better general area to look in to find the problem.
My challenge for 2021 is, then—learn and use the RINA model. I (humbly) believe it will actually make you a better network engineer.
On Important Things
I tend to be a very private person; I rarely discuss my “real life” with anyone except a few close friends. I thought it appropriate, though, in this season—both the season of the year and this season in my life—to post something a little more personal.
One thing people often remark about my personality is that I seem to be disturbed by very little in life. No matter what curve ball life might throw my way, I take the hit and turn it around, regain my sense of humor, and press forward into the fray more quickly than many expect. This season, combined with a recent curve ball (one of many—few people would suspect the path my life has taken across these 50+ years), and talking to Brian Keys in a recent episode of the Hedge, have given me reason to examine foundational principles once again.
How do I stay “up” when life throws me a curve ball?
Pragmatically, the worst network outage in the world is not likely to equal the stresses I’ve faced in the military, whether on the flight line or in … “other situations.” Life and death were immediately and obviously present in those times. Coming face to face with death—having friend who is there one moment, and not the next—changes your perspective. Knowing you hold the lives of hundreds of people in your hands—that if you make a mistake, real people will die (now!)—changes your perspective. In these times you realize there is more to life than work, or skill, or knowledge.
Spiritually, I am deeply Christian. I am close to God in a real way. I know him, and I trust his character and plans for the future. Job 13:15 and Romans 12:2 are present realities every day, from the moment I wake until the moment I fall asleep.
These two lead to a third observation.
Because of these things, I am grateful.
I am grateful for the people in my life—deeply held friendships, people who have spoken into my life, people who have helped carry me in times of crisis. I am grateful for the things in my life. Gratitude is, in essence, that which turns what you have into what you want (or perhaps enough to fulfill your wants). Gratitude often goes farther, though, teaching you that, all too often, you have more than you deserve.
So this season, whatever your religious beliefs, is a good time to reflect on the importance of all things spiritual, the value of life, the value of friendship, the value of truth, and to decide to have gratitude in the face of every storm. Gratitude causes us to look outside ourselves, and there is power (and healing) in self-forgetfulness. Self-love and self-hate are equal and opposite errors; it is in forgetting yourself, pressing forward, and giving to others, that you find out who you are.
Have a merry Christmas, a miraculous Hanukkah, or … just a joyful time being with family and friends at home. Watch a movie, eat ice cream and cookies, make a new friend, care for someone who has no family to be with.
To paraphrase Abraham Lincoln, most folks are just about as happy as they choose to be—so make the choice to be joyful and grateful.
These are the important things.
Pulling Back the Curtains
One of the major sources of complexity in modern systems is the simple failure to pull back the curtains. From a recent blog post over at the ACM—
Yegor describes three different reactions when a coder faces something unexpected when solving a problem.
Throw in the towel. Just give up on solving the problem. This is fairly uncommon in the networking and programming fields, so I don’t have much to say here.
Muddle through. Just figure out how to make it work by whatever means necessary.
Open the curtains and build an excellent solution. Learn how the underlying systems work, understand how to interact with them, and create a solution that best takes advantage of them.
The first and third options are rare indeed; it is the second solution that seems to dominate our world. What generally tends to happen is we set out to solve some problem, we encounter resistance, and we either “just make it work” by fiddling around with the bits or we say “this is just too complex, I’m going to build something new that simpler and easier.” The problem with building something new is the “something new” must go someplace … which generally means on top of existing “stuff.” Adding more stuff you do understand on top of stuff you don’t understand to solve a problem is, of course, a prime way to increase complexity in a network.
And thus we have one of the prime reasons for ever-increasing complexity in networks.
Yegor says being a great programmer by pulling back the curtain increases job satisfaction, helping him avoid depression. The same is probably true of network engineers who are deeply interested in solving problems—who are only happy at the end of the day if they know they have solved some problem, even if no-one ever notices.
Pulling back the curtains, then not only helps us to manage complexity, it can alos improve job satisfaction for those with the problem-solving mindset. Great reasons to pull back to the curtains, indeed.
Current Work in BGP Security
I’ve been chasing BGP security since before the publication of the soBGP drafts, way back in the early 2000’s (that’s almost 20 years for those who are math challenged). The most recent news largely centers on the RPKI, which is used to ensure the AS originating an advertisements is authorized to do so (or rather “owns” the resource or prefix). If you are not “up” on what the RPKI does, or how it works, you might find this old blog post useful—its actually the tenth post in a ten post series on the topic of BGP security.
Recent news in this space largely centers around the ongoing deployment of the RPKI. According to Wired, Google and Facebook have both recently adopted MANRS, and are adopting RPKI. While it might not seem like autonomous systems along the edge adopting BGP security best practices and the RPKI system can make much of a difference, but the “heavy hitters” among the content providers can play a pivotal role here by refusing to accept routes that appear to be hijacked. This not only helps these providers and their customers directly—a point the Wired article makes—this also helps the ‘net in a larger way by blocking attackers access to at least some of the “big fish” in terms of traffic.
Leslie Daigle, over at the Global Cyber Alliance—an organization I’d never heard of until I saw this—has a post up explaining exactly how deploying the RPKI in an edge AS can make a big difference in the service level from a customer’s perspective. Leslie is looking for operators who will fill out a survey on the routing security measures they deploy. If you operate a network that has any sort of BGP presence in the default-free zone (DFZ), it’s worth taking a look and filling the survey out.
One of the various problems with routing security is just being able to see what’s in the RPKI. If you have a problem with your route in the global table, you can always go look at a route view server or looking glass (a topic I will cover in some detail in an upcoming live webinar over on Safari Books Online—I think it’s scheduled for February right now). But what about the RPKI? RIPE NCC has released a new tool called the JDR:
Finally, the folks at APNIC, working with NLnet Labs, have taken a page from the BGP playbook and proposed an opaque object for the RPKI, extending it beyond “just prefixes.” They’ve created a new Resource Tagged Attestations, or RTAs, which can carry “any arbitrary file.” They have a post up explaining the rational and work here.
The EXPERIENCE HAS SHOWN THAT Keyword (RFC1925, Rule 4)
The world of information technology is filled, often to overflowing, with those who “know better.” For instance, I was recently reading an introduction to networking in a very popular orchestration system that began with the declaration that routing was hard, and therefore this system avoided routing. The document then went on to describe a system of moving packets around using multiple levels of Network Address Translation (NAT) and centrally configured policy-based routing (or filter-based forwarding) that was clearly simpler than the distributed protocols used to run large-scale networks. I thought, for a moment, of writing the author and pointing out the system in question had merely reinvented routing in a rather inefficient and probably broken way, but I relented. Why? Because I know RFC1925, rule 4, by heart:
Some things in life can never be fully appreciated nor understood unless experienced firsthand. Some things in networking can never be fully understood by someone who neither builds commercial networking equipment nor runs an operational network.
Ultimately, the people who built this system will likely not listen to me; rather, they are going to have to experience the pain caused by large-scale failures for themselves before they will listen. Many network operators do wish for some way to get their experience across to users and application developers, however; one suggestion which has been made in the past is adding subliminal messages to the TELNET protocol. According to RFC1097, adding this new message type would allow operators to gently encourage users to upgrade the software they are using by displaying a message on-screen which the user’s mind can process, but is not consciously aware of reading. The uses of such a protocol extension, however, would be wide-ranging, such as informing application developers that the network is not cheap, and packets are not carried instantaneously from one host to another.
A further suggestion made in this direction is to find ways to more fully document operational experience in Internet Standards produced by the Internet Engineering Task Force (IETF). Currently, the standards for writing such standards (sometimes mistakenly called meta-standards, although these standards about standards are standards in their own right) only include a few keywords which authors of protocol standards can use to guide developers into creating well-developed implementations. For instance, according to RFC2119, a protocol designer can use the term MUST (note the uppercase, which means it must be shouted when reading the standard out loud) to indicate something an implementation must do. If the implementation does not do what it MUST, subliminal messaging (as described above) will be used to discourage the use of that implementation.
MUST NOT, SHOULD, SHOULD NOT, MAY, and MAY NOT are the remaining keywords defined by the IETF for use in standards. While these options do cover a number of situations, they do not express the full range of options available based on operational experience. RFC6919 proposed extensions to these keywords to allow a fuller range of intent which could be useful to express experience.
For instance, RFC6919 adds the keyword MUST (BUT WE KNOW YOU WON’T) to express operational frustration for those times when even subliminal messaging will not convince a user or application developer to create an implementation that will gracefully scale. The POSSIBLE keyword is also included to indicate what is possible in the real world, and the REALLY SHOULD NOT is included for those times when an application developer or user asks for the network operator to launch pigs into flight.
Of course, the keywords described in RFC6919 may, at some point, be extended further to include such keywords as EXPERIENCE HAS SHOWN THAT and THAT WILL NOT SCALE, but for now protocol developers and operators are still somewhat restricted in their ability to fully express the experience of operating large-scale networks.
Even with these additional keywords and the use of subliminal messaging, improper implementations will still slip out into the wild, of course.
And what about network operators who are just beginning to learn their craft, or have long experience but somehow still make mistakes in their deployments? Some have suggested in the past—particularly those who work in technical assistance centers—that all network devices be shipped according to the puzzle-box protocol.
All network devices should be shipped in puzzle boxes such that only those with an appropriate level of knowledge, experience, and intelligence can open the box and hence install the equipment. Some might argue the Command Line Interface (CLI) currently supplied with most networking equipment is the equivalent of a puzzle box, but given the state of most networks, it seems shipping network equipment with a complex and difficult-to-use CLI has not been fully effective.