Hi, this is Kence Anderson, welcome back. In most of my science classes in college, there was a lecture and there was a lab. Most of the information was covered in the lecture but the lab was where you got your practice in. Designing AI brains is one of those things that requires a lot of practice. I've had calls with customers where I've listened and asked questions and then come up with a preliminary brain design in 15 minutes or 30 minutes or an hour. Sometimes people ask me, how did you know to ask those questions and how did you design the brain that fast? The answer, practice. The same will be true for you. If you practice designing brains you'll be able to design AI really well and really quickly even in high-pressure situations. To help you along. I'd like to give you four example problems that you can practice designing brains for. For each of those problems, I'll describe the actions, the goals, what's difficult about the problem, why you might want to use autonomous AI to control that system in the first place, and I'll give you some tips for sequences and strategies of skills. I'll also give you a few starter brain designs for each problem and then you can take it from there as you practice. Problem number one lunar lander. This problem comes from an old Atari video game but it's actually inspired by real spacecraft. I actually interviewed an engineer once who told me that this lunar lander game is pretty realistic. She was a person that worked on real planetary lander spacecraft and she told me that lunar lander mimics flight characteristics of real landing craft pretty well. It just has a few less rocket boosters on it. And then she described to me how The spacecraft that she worked on had 16 rocket boosters. The goal in this simple game-like environment is to land a spacecraft between the flags without crashing. You crash if the craft is traveling too fast when you hit the ground. You get more points for landing in the shortest amount of time. So that's a goal as well. The actions for the task, the actuators that you control are three rocket boosters, one main booster under the ship, and two side boosters one on each side of the ship. The main booster provides vertical thrust. The side boosters provide side thrust. Here's what's difficult about controlling the ship. The ship is inherently unstable like a helicopter or a drone. So controlling pitch and roll in real-time is difficult depending on the starting position of the craft. It also might depend on the starting velocity in various directions. In order to control the ship well, here's the information from sensors that the AI has access to. Horizontal position, vertical position, horizontal velocity, vertical velocity, angle, angular velocity, left leg contact whether the left leg is in contact with the ground and right leg contact, whether the right leg is in contact with the ground. Before I give you an example of any brain designs for lunar lander, let me point out a couple of things about skill sequences and strategies. One easy place to start is to think about landing. I think you'd agree that landing is a skill that's pretty different from any other skill involved in flying the craft. Okay. So what is landing? And when does landing happen? Landing is the final descent in the vertical direction to touch down on the surface? Yes, it's the very last thing you do in each scenario of the Lunar Lander game. So if there's a skill sequence, landing is the last skill in the sequence. All right, let me give you an example brain design that we built a long time ago to demonstrate concepts and lunar lander. The first concept is stability, stability is an important concept for all flying craft. If the greater concept is stability then the skill related to stability is to stabilize. Remember concepts tend to be expressed as nouns and skills which are specialized concepts for performing tasks towards specific goals tend to be expressed as verbs. So my first skill is to stabilize, the second skill is to move to center, and the third skill is the land. Okay, here's what the brain design diagram looks like for this AI. In this brain design, we arranged the skills as strategies in a hierarchy. We let the selector determine which strategy is right for each scenario. By defining each skill as a brain module, we are explicitly teaching each of these strategies. This has some added benefits. One, we can train and validate each skill separately. Two, in addition to the brain outputting the control actions for the spacecraft, the selector tells us which strategy it's selecting or how much it's using each strategy in the control over time. That means that this brain is telling us a bit about why it's making its decision because it's revealing the scenario that it thinks it's in and the strategy that matches. So if it calls the landing skill, the selector calls the landing skill then it's telling us that it thinks the landing craft is in the landing scenario, it's close to the ground perhaps or maybe it's traveling really quickly towards the ground. Which is why it's calling the landing strategy to make the control decision for the craft at that time. Take a look at this video of how the brain performed. >> What does it look like once we tell it to stay stable? All right, here it's trying to stay stable, it's still crashing but it's learned to stay stable more or less. All right, how about when we ask it to go to the center? Okay, now, it's trying to stay stable and it's trying to stay over the landing site. It's not always doing a great job, It's learning that's fine. So we can let that learn for a while and this is what it looks like more or less when it does this. It's funny when you actually get into the state it rarely lands because it's not trying to land, it's just trying to stay in the middle. Okay, now, how about when you've got all three? We've got all three together, how does it perform? Right, so the system's basically learned effectively how to play this game. >> All right, let's take a crack at modifying this brain design. This is the kind of practice that you'll do as well. Question, does it really make sense for us to allow the selector to choose the landing strategy at any time? I don't think so. Since landing is the final descent, it's the skill that you should use at the end of the scenario. I think it makes more sense as a sequence. So here's a picture of what the brain design looks like, where the hierarchy of strategies is a selection between stabilizing and moving to center and then landing is activated as the final stage of the sequence. This is a hybrid version of our functional and strategy design paradigms. The strategy block is from the design paradigm of strategies and the sequence of the strategy block and then the landing concept Is a sequence of functional skills. The brain design diagram for this design looks really similar to the original. So I'm going to include the task algebra that the selector would use. That's the logic that the selector needs to obey in its sequencing decisions. That's where you actually see the sequence. Okay, so now it's your turn. I want you to design one or more brains for lunar lander. Remember, you can also use perceptive concepts. You can use action or directive concepts like the ones we used and you can also use selectors like the ones we use but don't forget about experimenting with perception concepts. I'd love to see us post and explain some of our brain designs from this practice in the forums. That would be fantastic. Problem number 2, project Moab. Project Moab is a ball balancing robot designed to demonstrate end to end machine teaching. From selecting a use case that's the focus of course one of the specialization, machine teaching for autonomous AI to designing brains. That's the focus of this class all the way through building and deploying your AI brains. The project Moab device can fit on your desktop and its name is inspired by the balancing rock in Moab Utah. Moab uses an upward facing camera to detect the position and velocity of the ball on the plate. The position and velocity are described in terms of X and Y coordinates on the plate. Moab then uses three actuators to change the angle of the plate. The angle of the plate is described in terms of pitch and roll. While it may seem at first glance like project Moab is a simple device for toy problems. This is actually based on a very realistic set of devices and problems just like lunar lander is. The project Moab robot is inspired by the Stewart platform. It's a simplified version, Moab has three arms, a Stewart platform has six but the idea is still the same. The arms move around and that changes the pitch roll and yaw of the surface of the top of the platform. Remember Grant Bristow, our resident aerospace engineer from Bell flight that we've been following throughout this specialization. Well, when he was showing me the Bell 505 helicopter and explaining how it worked, I looked through a window and I saw this gigantic Stewart platform and it reminded me of the Moab. The Stewart platform there at the Bell Training Academy is used as a flight simulator. The arms move in ways that replicates scenarios for flight, maybe turbulence, maybe wind, maybe landing. And the pilot controls the simulation from inside the dome that's sitting on top of the Stewart platform. The first brain that you'll build in the third course of this specialization building autonomous AI is a single module. Single concept, one skill brain that learns to balance the ball on the plate. In the first week of the course, you'll also assess that brain after you build it. For this brain design lab, I want you to choose a task, design a task maybe something a little bit more complicated than balancing the ball on the plate. Simply keeping the ball from rolling off the plate. And I want you to design a brain for that task. You can use the perception, the computer vision in the Moab as a perceptive concept in your brain. But you also need to design some directive action concepts and maybe select your concepts depending on what skills you want the brain to perform. Post your designs in the forum and we can discuss and learn from each other. Problem number 3, heating, ventilation and air conditioning or HVAC. You've seen this problem before in course one and now we're going to design a brain to control the system. The goals of this system and most HVAC systems are to minimize energy costs while keeping temperature within comfort. That's directed by set point. Like when you set the thermostat at home and keeping carbon dioxide within legal limits. The system keeps the occupants comfortable by obeying the set points that they input and there's a tradeoff goals. It costs energy and money to heat and cool the building. And then there's the control strategies, the single control action for this system is to control the angle, the opening close of a damper, closing the damper recycles here. That's the first strategy, it's best to recycle air when the outside air is really cold or really hot and when energy is most expensive. That's when it's going to cost the most to heat and cool the building. Instead, we'd rather wait until the price of energy is cheaper or the weather is less extreme to heat or cool that outside air. The second strategy is to open the damper. This freshens the air, you see after a while, when you're recycling here, carbon dioxide builds up from the people inside breathing. And there's limits to how much carbon dioxide is allowed in the building to be healthy. The hardest part of this problem is reading the scenarios, energy price changes throughout the day. There's peak energy scenarios and off peak scenarios for energy prices. The outdoor temperature is cyclical. There's a pattern underneath it sure. Both the daily pattern and a seasonal pattern, but it's changing all the time. And then there's uncertainty sitting on top of those patterns. Your brain is going to have to figure out how to trade off the strategies to meet all the goals. With all this uncertainty in the scenarios, here's my basic brain design for you to start with. You can see this brain in action and play with the results and our machine teaching demo site. Now it's your turn, I want you to modify this brain design to make it different or more complex based on some aspect of the problem. Remember the goal is better performing autonomous AI. Problem number 4, warehouse logistics. We talk a lot about robots and machine control in this specialization. And I want to make sure to include this example to emphasize that autonomous decision making is really relevant to logistics processes and virtual systems as well. I've been asked to come in and design autonomous AI to help out in warehouses quite a few times. Warehouses are like batteries that store products for later when they're needed most. Without them, you have to make the exact amount of goods that everyone needed and ship them out to them immediately when they wanted to buy them. That's almost impossible. The goal is to maximize throughput in the warehouse. So we need to minimize the time it takes on average to fill a truck order with goods from the warehouse, goods come into the warehouse and then we decide where to store them. But when the trucks pull up to get the goods from the warehouse, how long does it take to get them out? That totally depends on how and where we stored them in the warehouse. The action is the storage location of each item that comes into the warehouse. This is a difficult decision to make because the demand for various goods has patterns but it's uncertain. Okay, let's talk about strategies for a minute. There's two strategies that I can immediately think of mostly based on what I see in the supermarket and the college job I had in a warehouse that might help us. I'm going to use them in my example brain design. One strategy is a consolidated strategy. Some goods last a long time and they're not needed very often, maybe when they are needed, they are bought in large quantities. I like to think of onions and potatoes in this category. In comparison to other produce. They last a really long time, so when a truck pulls up, it might ask for a lot of potatoes or a lot of onions. So we can go ahead and put those in one location in our warehouse, so that when the truck pulls up, you can just get a whole bunch of them really easily. Another strategy is a distributed strategy. Have you ever noticed when you go to the grocery store at least grocery stores in the United States where I live. And things like potato chips or salsa, are all over the grocery store, sometimes in front of almost every aisle. That's the distributed strategy. What the planners are saying is hey, I have an aisle for potato chips and salsa. But since it's something that so many people need so often I'm going to put it in front of lots of different places in my store, so that people can get it on their way to getting other things. Produce like lettuce in the warehouse might be produce which we might want to use a distributed strategy for. You place it in lots of different parts because trucks are frequently coming and asking for lettuce and you don't want to have to go all the way to one particular spot. If the truck pulls in over here, if the truck pulls in over here. Let me give you an example, look at this picture, of the brain design. I'm going to use these two strategies, consolidated and distributed in a hierarchy with a selector to learn which scenarios, are best to use for each strategy. These are pretty high level strategies. Remember, each of these strategy skills, has a big job to do. They have to decide once they're called, to determine the exact bins to put each product in. So, if one trip through the brain is deciding where to put all the lettuce from a particular incoming truck into the warehouse, then, the selector decides, I want to use the distributed strategy for lettuce. Then that distributed strategy concept, needs to decide for all the lettuce that's in that truck, which specific bins or areas in the warehouse, to put that lettuce in. I'm going to have to be very thoughtful about which technologies I want to use to execute a strategy like this. I might use optimization, to execute each of these strategies. So for example, if I have a selector, deep reinforcement learning is really, really good at doing these kinds of trade offs. That decides whether avocado should be stocked using the distributed or the consolidated strategy. Then maybe I use an optimization algorithm, to try combinations and find out how best to consolidate avocados in the warehouse by placing a certain amount in each place of the warehouse slot. So, I might consolidate the avocados over here or it might be best to consolidate them over here, or over here, or over here. Based on maybe predictions about incoming demand for avocados in the future. My warehouse simulator is going to give me feedback, on how long it will take to retrieve each of the items and which items each truck that comes, will need in order to fill its order. So the simulator is going to simulate the demand scenarios. How much avocados we're going to need, how much lettuce we're going to need, how much tomatoes are going to be picked up. And the amount of time it will take to retrieve the items based on how I stock them. Brain you previously decided to stock the tomatoes this way. So when truck number three pulls up, and ask for this order, it's going to take this amount of time to get the tomatoes because of how you stock them. Okay, your turn. Start by thinking about what other strategies you might employ besides the consolidated and the distributed strategy. Remember, I just came up with them by thinking about my own experience in a summer job, and me going to the shopping market every week. Try to design a brain, with these new strategies that you've come up with, in a few different ways to orchestrate those skills together. Post your brain design in the forums, and we can discuss it. [MUSIC]
