Fabric versus Network: What’s the Difference?

We often hear about fabrics, and we often hear about networks—but on paper, an in practice, they often seem to be the same thing. Leaving aside the many realms of vendor hype, what’s really the difference? Poking around on the ‘net, I came across a couple of definitions that seemed useful, at least at first blush. For instance, SDN Search gives provides the following insight—

The word fabric is used as a metaphor to illustrate the idea that if someone were to document network components and their relationships on paper, the lines would weave back and forth so densely that the diagram would resemble a woven piece of cloth.

While this is interesting, it gives us more of a “on the paper” answer than what might be called a functional view. The entry at Wikipedia is more operationally based—

Switched Fabric or switching fabric is a network topology in which network nodes interconnect via one or more network switches (particularly crossbar switches). Because a switched fabric network spreads network traffic across multiple physical links, it yields higher total throughput than broadcast networks, such as early Ethernet.

Greg has an interesting (though older) post up on the topic, and Brocade has an interesting (and longer) white paper up on the topic. None of these, however, seem to have complete picture. So what is a fabric?

To define a fabric in terms of functionality, I would look at several attributes, including—

• the regularity and connectiveness of the nodes (network devices) and edges (links)
• the design of the traffic flow, specifically how traffic is channeled to individually connected devices
• the performance goals the topology is designed to fulfill in terms of forwarding

You’ll notice that, unlike the definition given by many vendors, I’m not too interested in whether the fabric is treated as “one device” or “many devices.” Many vendors will throw the idea that a fabric must be treated as a single “thing,” unlike a network, which treats each device independently. This is clever marketing, of course, because it differentiates the vendor’s “fabric offering” from home grown (or built from component) fabrics, but that’s primarily what it is—marketing. While it might be a nice feature of any network or fabric to make administration easier, it’s not definitional in the way the performance and design of the network are.

In fact, what’s bound to start happening in the next few years is vendors are going to call overlay systems, or vertically integrated systems, for all sorts of things, like a “campus fabric,” or a “wide area fabric.” Another marketing ploy to watch out for is going to be interplay with the software defined moniker—if it’s “software defined,” is a “fabric.” Balderdash.

Let’s look at the three concepts I outlined above in a little more detail.

Topology Regularity

Fabrics have a well defined, regularly repeating topology. It doesn’t matter if the topology is planar or non-planar, what matters is that the topology is a regularly repeating “small topology” repeated in a larger topology.

In these diagrams, A is a regular topology; note you can take a copy of any four nodes, overlay them on any other four nodes, and see the repeating topology. A is also planar, as no two links cross. B is a nonplanar regular (or repeating) topology. C is not a regular topology, as the “subtopologies” do not repeat between the nodes. D is not a regular topology that is also nonplanar.

The regularity of the topology is a good rule of thumb that will help you quickly pick out most fabrics against most non-fabrics. Why? To understand the answer, we need to look at the rest of the properties of a fabric.

Design of the Traffic Flow

Several of the definitions given in my quick look through the ‘net mentioned this one: in a fabric, traffic is split across many available paths, rather than being pushed onto a smaller number of higher speed paths. The general rule of thumb is—if traffic can be split over a large number of ECMP paths, then you’re probably looking at a fabric, rather than a network. The only way to get a large number of ECMP paths is to create a regularly repeating topology, like the ones shown in A and B above.

Performance Goals

But why does the number of ECMP paths matter? Because—fabric performance is normally quantifiable in somewhat regular mathematical terms. In other words, if you want to understand the performance of a fabric, you don’t need to examine the network topology as a “one off.” Perhaps a better way to say this: fabrics are not snowflakes in terms of performance. You might not know why a particular fabric performs a certain way (theoretically), but you can still know how it’s going to perform under specific conditions.

The most common case of this is the ability to calculate the oversubscription rate on the fabric; whatn amount of traffic can the network switch without contention, given the traffic is evenly distributed across sources and receivers? In a fabric, it’s easy enough to look at the edge ports offered, the bandwidth available to carry traffic at each stage, and determine at what level the fabric is going to introduce buffering as a result of link contention. This is probably the crucial defining characteristic of a fabric from a network design perspective.

Another one that’s interesting, and less often considered, is the maximum or typical jitter through the fabric in the absence of contention. If a fabric is properly designed, and the network devices used to build the fabric don’t mess with your math, you can generally get a pretty good idea of what the minimum and maximum delay will be from any edge port to any other edge port on a fabric. Within the broader class of network topologies, this is generally a matter of measuring the actual delays through the network, rather than a calculation that can be done beforehand.

While some might disagree, these are the crucial differences between “any old network topology” and a fabric from my perspective.