Network scale is more than size

“Judge me by my size, do you?”

I’ve had several discussions with people over the years about the concept of scale in the world of network engineering. Most often, when network engineers think of a “large scale network,” they used to mention large service providers. Now they tend to think of some large cloud provider. But is scale really about size? I’m not much into the backflipping Yoda of the later Star Wars movies, but I would argue scale is much more about backflips than it is about being big.

So what is scale about? In the networking world, scale can be given the shorthand services x size. Standing in a huge data center with rows and rows of racks and blinking lights, it’s easy to forget about the services part of that equation.

A useful way to understand this is consider the services offered by a pair of networks, one large, and one small. The typical cloud provider’s network might contain thousands of nodes in a single data center — something more than 1000x10g (or 10,000x1g) ports on the edge is moderately sized in this world. What services does such a network — within the network itself — actually offer, though? There is some traffic engineering, to be certain, but very little quality of service beyond traffic engineering is probably configured. It’s easier for a large provider to manage all their quality of service in one place (through traffic engineering) rather than through multiple places, and with a plethora of queues.

What about applications and services? A useful case to consider is social media and search providers. In these networks, the application is often intimately tied to the network — but there are few applications, or they are tuned to the network as much as the network is tuned to the application. When you get to a certain scale, you can build container ships and the infrastructure that goes with them rather than relying on standard cargo ships.

At the other extreme is the small to mid sized network, whether enterprise or service provider. These networks don’t have a unified set of applications, nor can the applications necessarily be easily divorced from the network infrastructure. Security, for instance, isn’t left up to the application, but must be integral part of the network itself — the network must at least intelligently interact with the applications, even where limiting the intelligence of the network might be a good idea.

Which of these two networks is “larger scale?” The one with the simpler requirements and more ports, or the one with more requirements and fewer ports? Having worked on both, my impression is they have different challenges — but one doesn’t necessarily have bigger challenges than the other.

Taking the size versus scale concept to the extreme, someone once commented that small networks are easier to run because “a simple access list will give you the security you need.”

Sure. Until you realize the placement and content of that access list needs to be changed once an hour, and the security requirement is that the traffic to this particular server be blocked as early as possible from every port in the network. Suddenly complexity starts raising it’s head in even a simple little network that (perhaps) only needs one OSPF flooding domain. Now the problem is more easily solved with a virtual topology, and services enter into the picture as a scaling problem.

Before you discount small networks as “not an interesting scaling problem,” and stand in awe of the long line of data center racks as “large scale,” you need to consider the services side of the equation — you need to consider size and services. Scale is more than size.

Don’t let the size fool you.