[SOUND] Hi, this is Kence Anderson. In this video, I want to talk to you about deconstructing the problem. Two paradigms for orchestration. There's a saying that the best way to digest an elephant is one bite at a time. The same is true for the control of complex systems and the optimization of complex processes. Remember Grant Bristow from bell flight? He explained to me that autonomous flight systems need to be modular so that you can validate and certify each piece separately. He also showed me that flight control systems, even the ones that aren't autonomous are modular anyway. There's a lot of reasons to break down autonomous AI into modular pieces. One, modular systems are easier to test and certify. Two, modular systems are easier to maintain if one modular malfunctions or needs to be updated you don't have to rework the entire system. Three, different scenarios require different approaches, just like the three phases of the chess game or different operating regimes and manufacturing. Different skills and even different decision-making techniques may be required in different scenarios for effective autonomy. Four, learning systems, humans, and most AI systems alike learn better when they learn individual skills separately. Five, the very nature of teaching is to break down and sequence concepts into lessons. I knew this to be true, but for a long time I didn't have any standard method for breaking down problems, but I started to discover orchestration patterns as I designed more and more AI brains. The first two patterns that I discovered are the two paradigms for orchestration that I'll teach you in this video. There's many other patterns that I found as well. I'll teach you many more in course four. But every brain design pattern I've ever used, uses the same two paradigms of orchestration. The functional paradigm for sequences of skills and the strategy paradigm for hierarchies of skills. First, let's talk about the functional paradigm. There were a lot of situations where people brought me a problem that had many actions that needed to be taken, many decisions like even hundreds or thousands of decisions that needed to be made. This is how I discovered the functional paradigm. Let me give you two examples. The first is from an oil refinery. The second is from a liquefied natural gas plant. I designed a very complex brand to make the oil refining process more autonomous. Oil refining, like a lot of chemical manufacturing processes is continuous, which means that the inputs are constantly flowing in, and the outputs are constantly being made as opposed to a discrete process where you're making individual products. So each of the stages for each of the parts of this process are interdependent. What you do in one part of the process affects everything downstream and there's lots of decisions to be made, hundreds of them. As I paid close attention to the experts, they described that these hundreds of actions were actually partitioned into groups. I first discovered this when I asked them, hey, do you make all of these decisions simultaneously or can any of these decisions be treated as functionally independent? Functionally independent means that they depend on each other, but practically you can treat them separately. They told me that the decisions that they had sketched out on the whiteboard can be grouped into three groups. And when I asked them, how do you know that these decisions can actually be made independently of the decisions and the other groups, they said, well, that's how we do it currently. There are three separate teams that make decisions for each group separately from what's happening in the other groups even for this continuous process, and it works. This works in an oil refinery because in between the multiple stages of the process, the intermediate products, the current state of the chemical is stored in a tank until it's ready for the next stage of the process to work on it. So these tanks serve as physical buffers between the two parts of the continuous process. That's why you can treat them separately. Discrete manufacturing where you're making individual discrete products also uses buffers to separate parts of the process functionally. In manufacturing buffers provide a holding ground for the goods that are finished one step of the process but waiting for the next. The actions before the buffer and the actions after the buffer can often be treated independently. For other processes the experts couldn't tell me the physical or chemical reason why groups of actions were functionally independent. But they validated with much experience over long periods of time that they could be. Functional decomposition is a paradigm where you break down skills by separating mutually exclusive groups of actions. Functional decomposition allowed me to separate each group of actions in that oil refinery that were separated by a physical buffer of the tanks into a separate skill or concept. So now, instead of a brain with one skill that takes hundreds of actions. I have three skills that each take a more manageable number of actions or make a more manageable number of decisions at each step of the process. I know that this decomposition works because it's validated by subject-matter expertise and by physical reality. Now, some might say that's just one possible decomposition. How do you know that it's not an inferior one? How do you know that an AI brain might not be able to figure out a better decomposition? Well, let's remember the game of chess in the 12 strategies that alpha chess discovered by self practice. The self practice tells us that those 12 strategies are actually good groupings. Good ways to break up the decision space. Not just some human idea. This doesn't mean that every functional decomposition humans already use is a good one. That's why you should discuss with subject-matter experts and possibly even data scientists who can analyze data and see in that data different groups that might be functionally independent as you design your brain. Let me show you a picture of data that suggests multiple separate concepts or skills. Can you see the four skills? I designed another brain for a liquefied natural gas plant. This plant makes natural gas that many of us use to heat our homes. That plant had 1000 actions. So I asked the expert how many actions does an average operator take? How many knobs on that system would the operator actually need to manipulate to do a good job? Maybe 80% or even 90% of what an expert operator would do. He said 100 actions. Then I asked if you were teaching a new operator how many actions he'd need to run the plant. He said 12 actions. In that conversation, I was using the functional decomposition paradigm to separate actions into groups by level of impact. Nuance and expertise required to use them. Chess players do the same thing. Research shows that amateur chess players pay attention to far fewer chess pieces while deciding moves than experts and masters. See, there are many ways to cluster actions into mutually exclusive groups. Look for the axis that you can use as leverage to separate many actions into mutually exclusive groups. Let's diagram a brain that gives even more concrete example. Microsoft researchers built this brain to teach a robot to grasp and stack blocks. We've discussed this example before. The researchers designed a brain with five directive action concepts to execute skills, and a learned selector to supervise the concepts. Here's a list of the concepts. Reach, this movement extends the hand out from the body. Move, this movement sweeps the arm back and forth and up and down. Orient, this movement puts the robot hand in the right position to grasp the block. And grasp squeezes the fingers to grasp the block. Stack, this movement picks up the block and places it on top of another block. Each skill is a function that uses specific joints to perform a subtask. This is important because limiting the actions that each skill takes as it performs its function prevents the brain from having to explore many movements that couldn't possibly accomplish the goal. For example, orienting your hand around the block. Putting your hand in a position to pick it up involves rotating your wrist. Now, imagine if your arm is in the perfect position, and all you needed to do was turn your wrist to put your hand in position to grab the block, but you jerk your elbow. Now, your hand is in a position where you can't possibly grasp the block no matter how you turn your wrist. So, how do you know which sequences to use in a brain design? Let's look at some options for this robotic arm application. There are a few different kinds of sequences that make sense for arranging skills. Let's compare options and then discuss two different types of sequences. One simple skill arrangement is to complete the reach movement and then move, then orient, grasp, and stack. So that would look like this. Blocks over here, you complete the reach and then you move the entire reach and then you move. Remember reach is extending the hand out from the body, and move translates the hand laterally. When you reach first and then move the reach first, then move sequence looks a little clunky, but in certain situations it can totally get the job done. Here's a symbol that we can use to represent this kind of sequence. Finish the skill first completely then move on to the next. That's what this symbol means. Remember, these are sequences of skills not sequences of actions. What's the difference? Well, each skill generates its own sequence of actions to complete the skills. So when I reach each time step, I'm sending commands to the shoulder, wrist, elbow, and finger joints about how to complete that skill but the skill is a series of actions. But reaching and then moving is a series of skills. It's not a rigid sequence either especially if the skill is learned. For example, the reach skill will be able to reach from a variety of different starting points to a variety of different objects. Maybe avoiding a variety of different obstacles in the path between the starting point and the object. Okay, now, let's take a little bit more sophisticated view on how we can arrange the reach and move skills together in a sequence. If I allow the reach and move skill to alternate in any order, that frees up the brain to perform a lot more different kinds of movement patterns as it approaches the block. So instead of having to complete the entire reach and then move, I could reach, move, reach, move, reach, move, reach, move reach, move. And now I have a lot more flexibility to avoid obstacles and to make a smoother movement towards the obstacle. The first sequence of the reach and move skills is fixed. The reach skill is always performed before the move skill starts. The second sequence of reach and move skills is variable. It allows the brain to choose when to reach and when to move, basically how to order the reach and move skills. This symbol represents fixed sequences of skills and this symbol represents variable sequences of skills. So now that we've covered fixed sequences and variable sequences, two of the orchestration patterns in functional decomposition, I want to show you the third. The third orchestration pattern in functional decomposition is parallel execution. Sometimes you want to execute two functional skills in parallel. When this happens, two or more skills pass decisions to the brain output at the same time. Let me draw you a picture of what that might look like in a brain. If you have two skills operating in parallel, they both output their actions to the brain at each time step as the brain output. There's no selection mechanism between them because they're both active at all times. In order to do this, you have to make sure that there's no duplication in the actions between the two functional skills. Each of the skills that are executed in parallel must have a completely mutually exclusive. That's a completely separate set of actions. To make this work for a robotic arm, we have to address the shoulder joint. So, both the reach skill, shoulder, elbow, wrist. And the move skill, shoulder use or actuate, take an action to control the shoulder joint of the robot. You want to perform parallel execution of reach and move, executing the reach and move skills simultaneously. So we need to assign the shoulder joint to one, but not the other of the skills. So let's redefine the reach skill as using the elbow and wrist only like this. So now reach looks like this. The reach skill manipulates the wrist and elbow, and the move skill operates the shoulder joint. No duplication. Now, we can run both of these skills in parallel. When we do this, we can produce a completely smooth motion. When moving the hand from point to point using the reach and move skills together in parallel execution. I've given you some options for how you can orchestrate the first two skills, reach and move together. Can you think of ways to modify the orchestration for any of the other skills? Now, let's talk about the second paradigm for orchestration in AI brains. Strategy. The functional decomposition paradigm separates different actions into different skills. The strategy decomposition paradigm uses the same actions for all skills, but instead, strategy separates skills by scenarios where each strategy is most effective. We talked a lot about the different phases of the chess game in course one. Let's review. The chess opening or the first phase of chess has a goal of surviving and moving on to the next phase. The mid game has a goal of gaining advantage or a leg up on the opponent. The end game has a goal of mating which is trapping or pinning the opponent's king. Because each of these scenarios have completely different goals. Each of these phases have a different set of strategies that are used during each chess scenario. Opening strategies for the opening phase. Mid game strategies for the mid game and mating strategies for the end game. While it's technically possible to try to use mid game strategies during the end game or attempt endgame strategies during the mid game. It doesn't make much sense because these scenarios have different goals, which makes endgame strategies much more effective during the end of the game and mid game strategies a lot more effective during the mid game. Let me give you an industrial example to make strategies more concrete. I've designed several brains for mining companies to control Gira Torey rock crushers. These rock crushers are basically huge metal cones. I'm talking about multiple stories tall. A conveyer dumps rocks into the cone. There's a metal piece inside that gyrates and smashes the rocks up against the steel plates on the side of the cone and the crushed rock comes out the bottom. If the crushed rock is small enough, if it meets the size criteria, the finest criteria for the crushing operation, then it passes through a shaking sieve and moves on to the next phase of the mining operation. If the rocks are not small enough yet, they get recycled back into the crusher until they're small enough to pass through the sieve. Every company that I've ever talked to about Gira Torey rock crushers tells me about the same two strategies that operators and engineers use to control the crushers. Strategies are labeled courses of action that are particularly effective in specific scenarios, strategies are specific kinds of skills that are orchestrated into hierarchies. Using the strategy decomposition paradigm, the first strategy is to choke the crusher. That means we stuff the crusher full of rocks, chock full of rocks. It's so full that the compression can crush rocks even large hard rocks first time around. So the choking strategy works really well for large hard rocks. It's also very efficient because it works the first time recycling, very little recycling. But there's a drawback, it's slow because the crusher is stuffed so full, the rocks take longer to come out of the crusher, just like a really full funnel. So if you choke the crusher or use the choke strategy for softer small rocks, then you're really just slowing down the process because the crusher is full and it takes longer for the rocks to exit. But they were probably going to get crushed anyway by the strategy that I'm going to talk about in a second. There's always a tradeoff of strategies remember that, the second strategy is to regulate the crusher, that means to fill the crusher maybe half or two thirds or to three quarters of the way full. This way the crusher is not stuffed full which allows the rocks to exit the crusher more quickly. But there's a trade off. When you regulate the crusher, you can't crush very large or very hard rocks. So the regulate strategy is perfect for softer smaller rocks. Let's draw a diagram of these two strategies in an AI brain. As always our brain has input and output. I usually highlight these input and output nodes with yellow. We have two strategies and we have a selector that's going to select between the strategies. In other words, at each decision point the selector will select which strategy gets to make the final decision. These strategies are arranged in a hierarchy. The selection mechanism is at the top of the hierarchy and the action concepts are at the bottom of the hierarchy. Here's a symbol that we're going to use to show the hierarchies that strategies operate in. The two strategies that we have to choose from when operating the rock crusher, choking and regulating are shown in the brackets separated by commas. That's the bottom level of the hierarchy. At the top of the hierarchy is a selector skill that chooses when to use each strategy, when to choke the crusher and when to regulate the crusher. All right, so as you're deconstructing the problems that you're going to solve with autonomous AI you have to decomposition paradigms to work with. The functional decomposition paradigm separates actions into different skills and then arranges them in sequences or parallel execution. The strategy decomposition paradigm, separate skills by the scenarios where they will work best. [MUSIC]
