ooda loop

Act!

Part 1: Getting Inside the Loop
Part 2: Orientation
Part 3: Decide!

Once you’ve observed, oriented, and decided, it’s time to act. This might seem like a minor concept, but it’s actually really, really hard to act in a lot of situations. There are two elements here — the first is our willing suspension of belief, and the second is the doubt storm. Let’s talk about these two.

The willing suspension of belief. To find an example here, I’m going to fall back on my training in self defense. When you first find yourself in any situation that is “bad,” your first line of thought is going to be “this isn’t really happening,” or “why would this person want to hurt me?” In the same way, when your network is failing or under attack, the easiest thing in the world is to disregard the loop, roll over, and go back to sleep. Why would anyone attack my network? Why would this bug be hitting my control plane? Like Scrooge faced with a ghost, we say, “there’s more gravy than grave about you.” And this is a grave mistake. There’s a reason you’ve gone through all the trouble of thinking through what you’re going to observe, how you’re going to orient, and then thinking through what you’re going to decide.

So you can act when you need to.

You see, we’re all humans, and we all get trapped in the moment of conflicting emotions, the moment of high pressure and high adrenaline. It’s precisely at this moment that you don’t want to be deciding what to do, and deciding to do it. If you’re here, someone, or something, has gotten inside your loop, and you’re about to fail.

[tr-shareit text=”Why would anyone attack my network? Why would this bug be hitting my control plane? Like Scrooge faced with a ghost, we say, “there’s more gravy than grave about you.” And this is a grave mistake.” sites=”twitter,facebook,google” align=”left”]Why would anyone attack my network? Why would this bug be hitting my control plane? Like Scrooge faced with a ghost, we say, “there’s more gravy than grave about you.” And this is a grave mistake.[/tr-shareit]This rolls right into the second reason above — the doubt storm. The internal conversation sounds something like this, doesn’t it?

“What if I do this and it shuts down the CEO’s phone call?”
“What if I don’t, and the entire network fails?”
“What should I do?”
“WHAT SHOULD I DO?”

If you’ve followed the loop, and you’ve planned ahead, follow the plan. It’s that simple — just do it. Hone your skills, know your network, know your monitoring points, know what you’re looking at, know your plan, and do it.

A lot simpler said than done, I know. But the entire point here is to plan when you have time so you don’t have to fall to the doubt storm when you don’t have time to plan. The point is to work off what you know, rather than what you feel at that one moment. There is a metaphysical relationship of faith to reason here, as well — but don’t get me started on philosophy, or we’ll be here forever. 🙂

The bottom line is — make your plan, and then trust your plan.

Observe, Orient, Decide, and Act.

And that’s a wrap. Or should we say a loop?

The OODA loop is covered in some detail in The Art of Network Architecture, available wherever fine books are sold (because if they don’t sell my books, then they don’t sell fine books — see how that works?).

OODA Part 1: Getting Inside the Loop
OODA Part 2: Orientation
OODA Part 3: Decide!
OODA Part 4: Act!

Tags: |

Decide!

Last week we talked about orienteering — using models and information to orient ourselves to what’s going on in the network. This is part of the OODA loop, which we talked about two weeks ago. This week, let’s look at the next step — decide.

In fact: Decide! Now! How many hours have you spent thinking through what to decide? Which car to buy, which house to buy, which vacation to take, which… It seems like our life is a continuous stream of decisions through which we must dig deeply to make a choice. Sometimes it makes you want to replace your entire outfit with grey and black. Everything. Actually, we should feel blessed to have so many decisions — at least we’re not considering “eveningvear…” (note the ever fashionable flashlight).

But the last place you want to be is in the middle of a major network outage or attack, spending hours deciding — what was it we were deciding? By the time you get to the fifth pizza and the tenth box of bonbons, maybe you’ve forgotten what you are sitting in that “war room” for. There is another alternative, of course.

Decide what you’re going to decide before you have to decide.

Okay, that might sound a little “round about,” but let’s take an example from self defense training to illustrate. There are four levels of in the cycle of self defense. White — you’re unaware of your environment. You should never be here. Yellow — you’re aware of your environment, actively thinking about potential threats. Red — you’ve spotted what you think is a threat, and are preparing to react. Finally, black — you believe you are actually under attack, and are actively reacting to that attack. What’s important here, from the perspective of the OODA loop, is that between red and black you need to make a plan. You need, for instance, to look for cover and/or concealment. You need to think about how you’re going to exit the situation, whether it’s pulling off on the shoulder and driving away, or finding the exit, or…

To give another example — when you’re sitting there listening to the safety briefing before takeoff for the nine millionth time, what are you doing? Are you looking at where the exits are, and thinking about how you can get there when that person two aisles in front of you just shoved the largest suitcase you’ve ever seen under the growing seat in front of them? Or are you playing Candy Crush?

In all these situations, you should be deciding before you have to decide so you can shorten the OODA loop.

[tr-shareit text=”Decide what you’re going to decide before you have to decide.” sites=”twitter,facebook,google” align=”left”]Decide what you’re going to decide before you have to decide.[/tr-shareit] In a network context, you need to think about things like:

  • Where would you put a filter to block this particular type of traffic?
  • Which parallel links would you remove to kill off that positive feedback loop that’s keeping your routing protocol from converging?
  • What servers can you shut down for a time while you’re trying to figure out why the data center fabric has become so hot all of a sudden?

All of these decisions are choices you can make before the action starts — before you actually have to decide to do something. In other words decide what you need to do, so that when it comes time to actually do it, you’ll have a plan in place.

The OODA loop is covered in some detail in The Art of Network Architecture, available wherever fine books are sold (because if they don’t sell my books, then they don’t sell fine books — see how that works?).

OODA Part 1: Getting Inside the Loop
OODA Part 2: Orientation
OODA Part 3: Decide!
OODA Part 4: Act!

Tags: |

Orientation

Quick — can you OODA? Last week we talked about the general idea behind the OODA loop; this week we’ll cover one more step; next week we’ll cover the last two steps, and then, in the last post, we’ll review and wrap up.

Orient is the second step: once you’ve made a set of observations, you need to decide what it is you’re actually observing. To help this make sense, let’s take a look at a simple optical illusion — you might have seen it before.

perfectsquares

Do the blue squares look square, or… ?? If you’re like most people, the squares don’t look square at all — but they are. Remember the blue or gold dress? In both of these situations, we face the same sort of problem: our ability to perceive is often influenced by the context.

This doesn’t, as some people try to say, mean that our senses are all just a jumbled up mess, and the entire world is disconnected from our brains — you must be careful in life not to make the hard or odd case the rule by which all other cases are measured. Every measurement system has its limits; that doesn’t mean the measurement is useless or generally untrustworthy.

So what we must do, as network engineers, is to learn to figure out when the context matters, and when the context is simply messing us up. To separate the blue squares from the lines in the background, so to speak. How do we do this?

First, understand the operation of the network, protocols, and applications at a theoretical level. Reaching beyond the command line, and into the actual operation of the devices in the network — understanding how a router forwards packets, or how OSPF actually builds and processes packets, can make a huge difference in your ability to orient yourself to what you’re observing.

Second, learning and applying models is a huge help. The only reason you probably have trouble with the optical illusion above is that the boxes appear close enough to being squares that you immediately think they must, in reality, be squares. It’s easy enough to verify they are, actually, squares, but if we didn’t have the expectation of seeing squares there in the first place, we wouldn’t have suspected there was an illusion in play here. We have a “model of a square,” in our heads, and when see things that are close to that model, we try and make the object fit. Sometimes this works, sometimes it doesn’t. Now imagine someone who has only ever seen squares on seeing a rectangle for the first time. Try as they might, they can’t make the object fit into their model. This isn’t a problem with the model, or the object, it’s a problem with the person’s “model collection” in their head.

So it’s important to know a wide array of models into which any problem can fit — or, in the networking world, a wide array of models you can use to “see” protocol, application, device, and network operation. Each additional model you add to your “mental model set,” allows you to orient yourself that much faster.

This entire process is much like what I learned in orienteering in my younger days. First, get the map pointing north. Then, find the features on the map that match where you are, and work from there to the destination, feature by feature. Not orienting the map is failing to separate the background from the information. Not being able to see the surrounding area is failing to collect the information necessary to match the map to the reality. Not knowing the symbols on the map is failing to have enough mental models to make the match between map and reality happen.

All three are important in the orientation phase — so as network engineers, we need to try and gain all three skills.

The OODA loop is covered in some detail in The Art of Network Architecture, available wherever fine books are sold (because if they don’t sell my books, then they don’t sell fine books — see how that works?).

OODA Part 1: Getting Inside the Loop
OODA Part 2: Orientation
OODA Part 3: Decide!
OODA Part 4: Act!

Getting Inside the Loop

Metadata doesn’t just apply to data science or protocols — it applies to engineering life. Think about the concept of epistomology — the study of how we know what we know — or the concept of hermeneutics — the study of how we understand communication — and you can quickly see that stepping outside what we are doing to examine how we are doing it is a common human experience (see Lewis’ Meditation in a Tool Shed as another instance).

But how does this apply to the engineering life? It’s called process — now, before you click off the page, scurrying away in shock, process isn’t a bad thing. In fact, process can be a good set of “guard rails” in the way we live our lives, something to remind us not to run off the road (like positive thinking signs), or even physically/mentally “bump” us in the right direction.

This week I’d like to kick off a short series on one process I learned in the US Air Force, and have used in many ways over the years — the OODA Loop. Originally developed by USAF Colonel John Boyd, and designed to help pilots deal with quick decision making in life-or-death situations, I’ve found this little model, or process, taught to us airfield folks as one of the bits of “combat training,” if we were ever deployed in the line of fire. What is the OODA loop? It’s four steps:

    • Observe
    • Orient
    • Decide
    • Act

For this series, I’d like to work through each of these four steps, thinking through what each one might mean in networking. I’ll generally apply the concept to network security, but it can be applied to network management, deployment plans, and a host of other areas.

Let’s begin with observe.

Just open your eyes, right? Not really — especially not in the world of networks engineering. First, what should you observe, and where should you observe it? In some cases, this is the most important question to ask, and the hardest to answer. Should you measure the average traffic flow across specific points in the network? The average jitter across specific points? The average delay? The number of routes in the routing table? The rate at which the routing table changes?

I would suggest the right answer is — all of the above.

But of course you can’t measure all of it everywhere on the network all the time, so you must decide where and how to measure so you will get a good “feel” for the overall operation of the network on a day-to-day basis.

There is a second point hidden in observe, however — how do you know what you’re observing unless you record? As the old saying goes, “if you didn’t write it down, it didn’t happen” — and nothing is truer than this in the world of observation. There’s no point in knowing what’s happening right now unless you know what has happened in the past.

So the first step in the OODA loop is to decide where, how, and what to observe — and to record it in a way that makes it easy to reference in the future.

The OODA loop is covered in some detail in The Art of Network Architecture, available wherever fine books are sold (because if they don’t sell my books, then they don’t sell fine books — see how that works?).

OODA Part 1: Getting Inside the Loop
OODA Part 2: Orientation
OODA Part 3: Decide!
OODA Part 4: Act!

Tags: |