Hedge 152: Joel King on the network and DevOps

DevOps, SecDevOps, GitDevOps—stick DevOps on the end of anything, and it will sound cool, generation FOMO in thousands (maybe millions). What does DevOps really mean to network engineers, though? In this episode of The Hedge, we discuss examples of how the Three Ways, (described in Part One of The DevOps Handbook) of Flow, Feedback, and Continual Learning with Joel King, a leading light in this field.


Hedge 150: Micah Beck and Universal Broadband

What would the Internet look like—or what kinds of services would need to be developed and deployed—to make boradband class service available to every user? What could this kind of development do to drive entire societies forward? Micah Beck, from the University of Tennessee, joins Tom Ammon and Russ White to discuss universal broadband on this episode of the Hedge.


Controversial Reads 091022

After decades trailing the rest of the world in leading-edge chip making, Chinese sand stamper Semiconductor Manufacturing International Corporation (SMIC) has quietly got into the 7nm business. That’s a huge and unexpected leap. Has the West’s embargo of the latest fab furniture failed?

Although the United States and China are not engaged in traditional warfare, they are engaged in a war of ideas, trade, and technology, especially in semiconductor hegemony, where both sides are battling for supply and advancement.

Trade was global, the world was inextricably connected, and your job’s in China now but you should thank us, actually, because everything is cheap and fast and out-of-work factory workers can simply learn to code.

Last month, CFR issued the report of a new task force, “Confronting Reality in Cyberspace: Foreign Policy for a Fragmented Internet.” (I was project director for both reports.)

Nationalism has come to software. While downloading TikTok or WeChat onto your cell phone isn’t quite tantamount to installing Huawei equipment in your local cell tower, all indications suggest that a software geopolitical divide has arrived and won’t be going anywhere.

These Emails Show How the Biden Administration’s Crusade Against ‘Misinformation’ Imposes Censorship by Proxy

On July 16, 2021, the day that Joe Biden accused Facebook of “killing people” by failing to suppress misinformation about COVID-19 vaccines, a senior executive at the social media platform’s parent company emailed Surgeon General Vivek Murthy in an effort to assuage the president’s anger.

Recently, India, home to more than 1.3 billion people, withdrew its data protection bill—signaling yet another setback in global efforts to protect people’s data privacy.

Cloudflare takes the law into its own hands by blocking Kiwi Farms

The responsibility for policing the internet has once more come to the fore as network security provider Cloudflare felt compelled to block Kiwi Farms for allowing targeted threats.

BGP Policies (Part 3)

At the most basic level, there are only three BGP policies: pushing traffic through a specific exit point; pulling traffic through a specific entry point; preventing a remote AS (more than one AS hop away) from transiting your AS to reach a specific destination. In this series I’m going to discuss different reasons for these kinds of policies, and different ways to implement them in interdomain BGP.

There are many reasons an operator might want to select which neighboring AS through which to send traffic towards a given reachable destination (for instance, 100::/64). Each of these examples assumes the AS in question has learned multiple paths towards 100::/64, one from each peer, and must choose one of the two available paths to forward along.

In the following network—

From AS65001’s perspective

Assume AS65001 is some form of content provider, which means it offers some service such as bare metal compute, cloud services, search engines, social media, etc. Customers from AS65006 are connecting to its servers, located on the 100::/64 network, which generates a large amount of traffic returning to the customers.
From the perspective of AS hops, it appears the path from AS65001 to AS65006 is the same length—if this is true, AS65001 does not have any reason to choose one path or another (given there is no measurable performance difference, as in the cases described above from AS65006’s perspective). However, the AS hop count does not accurately describe the geographic distances involved:

  • The geographic distance between 100::/64 and the exit towards AS65003 is very short
  • The geographic distance between AS100::/64 and the exits towards AS65002 is very long
  • The total geographic distance packets travel when following either path is about the same

In this case, AS65001 can either choose to hold on to packets destined to customers in AS65006 for a longer or shorter geographic distance.
While carrying the traffic over a longer geographic distance is more expensive, AS65001 would also like to optimize for the customer’s quality of experience (QoE), which means AS65001 should hold on to the traffic for as long as possible.

Because customers will use AS65001’s services in direct relation to their QoE (the relationship between service usage and QoE is measurable in the real world), AS65001 will opt to carry traffic destined to customers as long as possible—another instance of cold potato routing.
This is normally implemented by setting the preference for all routes equal and relying on the IGP metric part of the BGP bestpath decision process to control the exit point. IGP metrics can then be tuned based on the geographic distance from the origin of the traffic within the network and the exit point closest to the customer.

An alternative, more active, solution would be to have a local controller monitor the performance of individual paths to a given reachable destination, setting the preferences on individual reachable destinations and tuning IGP metrics in near-real-time to adjust for optimal customer experience.
Another alternative is to have a local controller monitor the performance individual paths and use MPLS, segment routing, or some other mechanism to actively engineer or steer the path of traffic through the network.

Some content providers may directly peer with transit and edge providers to reach customers more quickly, to reduce costs, and to increase their control over customer-facing traffic. For instance, if AS65001 is a content provider that transits traffic through [65002,65005] to reach customers in AS65006. To avoid transiting multiple autonomous systems, AS65001 can run a link directly to AS65005.

In some cases, content providers will build long-haul fiber optics (including undersea cable operations, see this site for examples) to avoid transiting multiple autonomous systems.

While the operator can end up paying a lot to build and operate long-haul optical links, this cost is offset is offset by decreasing paying transit providers for high levels of asymmetric traffic flows. Beyond this, content providers can control user experience more effectively the longer they control the user’s traffic. Finally, content providers can gain more information by connecting closer to users, feeding into Kai-Fu Lee’s virtuous cycle.

Note: content providers peering directly with edge providers and through IXPs is one component of the centralization of the Internet.

A failed alternative to the techniques described here was the use of automatic disaggregation at the content provider’s autonomous system borders. For instance, if a customer connected to a server in 100::/64 by sending traffic via the [65003,65001] link, an automated system will examine the routing table to see which route is currently being used to reach the customer’s reachable destination. If traffic forwarded to this customer’s address would normally pass through one of the [65001,65002] links, a local host route is created and distributed into AS65001 to draw this traffic to the exit connected to AS65003.

The theory behind this automatic disaggregation was that the customer will always take the shortest path from their perspective to reach the service. This assumption fails, in practice, however, so this scheme was ultimately abandoned.

Weekend Reads 091021

At some point, don’t be surprised if IBM ends up acquiring Nutanix. But not now, with Nutanix having a nearly $8 billion market capitalization and possibly commanding a $10 billion acquisition cost.

Some of the most successful and lucrative online scams employ a “low-and-slow” approach — avoiding detection or interference from researchers and law enforcement agencies by stealing small bits of cash from many people over an extended period.

If you were hit by ransomware, you are part of the rapidly growing number of organizations that have had to decide how to respond — legally, quickly and often quietly.

As the pandemic unfolded in spring 2020, many Americans saw their lives swiftly reshaped by stay-at-home orders, school closures and the onset of remote work.

With increased demands placed on home internet connections and the nation’s internet infrastructure during the pandemic, the quality and affordability of home internet connections became a focus for users on several fronts.

The graphics card inside your computer is a powerful tool for gaming and creative work, but it can also potentially serve as a Trojan horse for malware.

Decentralized solutions, in our case, which come with the ambitious promise of providing everything their centralized counterpart can provide but without centralized points of failure and regulations. In our previous article, we enumerated several advantages associated with decentralized domain names.

With solution providers such as Unstoppable Domains or Handshake, and blockchain technology-friendly browsers, such as Brave, that are more than happy to assist on the implementation front, decentralized alternatives to the traditional Domain Name System has been receiving more and more attention lately.

A substantial amount of DNS community discussion on the topic of DNS Abuse is focused on defining what is or is not DNS Abuse.

I recently looked up a specialized medical network. For weeks following the search, I was bombarded with ads for the network and other related services: the Internet clearly thought I was on the market for a new doctor.

Expect to hear increasing buzz around graph neural network use cases among hyperscalers in the coming year. Behind the scenes, these are already replacing existing recommendation systems and traveling into services you use daily, including Google Maps.

A group of academics has proposed a machine learning approach that uses authentic interactions between devices in Bluetooth networks as a foundation to handle device-to-device authentication reliably.

Of course, the implications for such potential misdirection vary according to the nature of the service. So, let’s get personal. What about my bank? When I enter the URL of my bank, how do I know the resultant session is a session with my bank?

The U.S. Cybersecurity and Infrastructure Security Agency (CISA) on Monday added single-factor authentication to the short list of “exceptionally risky” cybersecurity practices that could expose critical infrastructure as well as government and the private sector entities to devastating cyberattacks.

Simple Mail Transfer Protocol or SMTP has easily exploitable security loopholes. Email routing protocols were designed in a time when cryptographic technology was at a nascent stage (e.g., the de-facto protocol for email transfer, SMTP, is nearly 40 years old now), and therefore security was not an important consideration.

History of Networking: Scott Bradner and the Early Internet at Harvard

Scott Bradner was given his first email address in the 1970’s, and his workstation was the gateway for all Internet connectivity at Harvard for some time. Join Donald Sharp and Russ White as Scott recounts the early days of networking at Harvard, including the installation of the first Cisco router, the origins of comparative performance testing and Interop, and the origins of the SHOULD, MUST, and MAY as they are used in IETF standards today.