Hi, I'm Juan Vergara excited to have you back. A simulation team has performed the necessary simulation validation steps to ensure the Lunar Lander sim works as expected. Then they uploaded the simulation to Azure. We are now ready to start machine teaching experimentation. Let's do it. As already discussed, the decomposition suggested for Lunar Lander by the Solution Architect contains four main concepts and one selector. The four trained concepts are move right, move left, move down, and stabilize, as well as the stop engines program concept. Then we have a selector to select the best strategy at any given point in time. The first test towards testing the decomposition is prototyping a couple of the concepts to validate the brain can succeed at the task. Always start machine teaching experimentation with simple initial experiments an overly complexes start can make it too difficult to understand what is helping and what is not. No matter if your brain succeeds or fails a simple start will help you set solid foundations. With a solid foundation, you can then expand and grow towards the final brain inkling file that is effective at teaching the brain to land safely. For this decomposition, we decided to start validation with two concepts, stabilize and move right. We expect that the move left concept will train similarly to the move right concept, this assumption is founded on the symmetry between both controls. Also, we expect move down to be a simpler concept, than move right or move left since the main power source points upwards once the ship has been stabilized. To build the brain, you should have a file attached to this video with the pieces you can copy and transfer into the Bonsai UI. We will be using the visual editor view so that we can focus on the problem definition and not the inkling language. Open the Bonsai preview website and click over create brain. Then select Empty brain and call your brain Lunar Lander Experimentation. Disregard the training error for now. You want to unselect visual editor first so that we can add another bucket that we are going to rely on. Insert using number so that we can later define the Boolean variables on later state configuration. Let's enable visual editor again. The first thing you will need to do is to drop a learned concept. Go to the list of concepts and drag the learn icon to the center of our brain grab. Name this learned concept stabilize. You will notice the simulation box appears to the left after selecting on Stabilize. Click over the simulation called my simulator, and modify the name to LunarLanderSim. Now it's time to define the inputs to be received from the simulation. The state's selects the drop-down menu next to State and click over new type. For prototyping destabilized concept, we first need to define all provided states by the sim to the brain. A structure called sim-state is created and all the variables provided by the simulation are added to the list. First, we have the two positions of the spaceship. Then we can identify the two velocities of the spaceship. Thirdly, we can identify both the angle as well as the angular speed. Last but not least, we have two flags that are used to indicate if the ship has effectively landed on the ground. As added parameters, we have both the gene reward and gym terminal. We will be using the gym terminal to identify, once the ship has crashed. Transfer the state parameters from the attached instructions file. Then click, Save and close. Note that sometimes the states provided by simulations do not relate to sensors that the brain will have access to once deployed. All the states that will not be accessible during deployment should not be observed by the brain. Otherwise, you are injecting a dependency during brain training that will not be satisfied during deployment. From the state, describe the only two that are unrelated to sensors are the ship crashed and ship landed flags. Thus, we go ahead and remove them from the observable state. To do so, click over the top box, that is over our stabilized concept, on the drop-down menu, next to graph input, select new type. Here paste the observable state struct available on the attached instructions file. Then transfer all these states that exist within the sim state struct. Last but not least, we remove the ship crash and ship landed variables. We are good to save and close. You'll notice a circle appearing at the top with a right arrow inside. This indicates an implicit state transform from sim-state to observable state. In other words, the simulation provides several inputs to the brain yet not all will be needed to query actions. Once the brain is deployed. Once the states have been defined, we are ready for the next structure, the action space. For our simulation, the minimum and maximum power, are minus 1 and 1 for the main engine and 0-1 for the side engines. Click over the simulation again. Go to action underneath state, and click on the drop-down menu. Then hit the new type button. On the center of the screen, paste the brain action struct, you will find within our instructions file. You will notice our brain is complaining about the lack of definition for ThrottleMin and ThrottleMax. Let's add those next. For now, save and close the brain action struct. Then click over this stabilized concept. Then click over "Global". If you see the LunarLanderSim has been selected, exit by clicking on the left arrow. Then it's time to add the constants. Within the constants section, click the "Plus" button and enter ThrottleMax selecting a value of one. Then add the second constant. ThrottleMin should have a value of minus 1. With the previously defined ingredients, we are ready to define the training graph for our first experiment. To ensure a simple initial structure, we decide to prototype the stabilized concept first by itself. The stabilized concept has three objectives as defining the AI spec. Avoid crashing, avoid rotation, and drive the spaceship to stop position in all directions. We go ahead and translate those three rules into their respective inkling objectives. Click over the "Stabilize" concept and select the "Plus" symbol next to go. Then transfer the goal definition from the instructions file attached to this video. You will see that we have actually four goals in inkling. One avoid crash goal and three minimize goals, two independent ones for translation both horizontally and vertically, as well as the last one to minimize the angle of the ship. Hit "Save and close". It is always helpful to add meaningful notes so you keep track of the changes made throughout the experimentation process. The first few words of any notes are actually displayed next to the brain version on the left pane. For now, click on the "Notes" tab and enter the following definition, stabilize plus 3 minimize goals and one avoid crash. See how the text is visible on the left pane. With all the pieces in place we're good to start training. You should see the train botton is now green. Clicking the green button we start our training session. We'll see you back once this session is finished training. Please go ahead and do this on your own now. You will have to select the Lunar Lander Simulation. In your case, you should only see one item. Please select that version. Once you have started training, you will see it takes some time for this session to start. You can click on the "Errors and Output" tab to see if everything is working as expected. After several steps are sampled, the brain will select an algorithm and will start the training session. Given the simulation speed for Lunar Lander, the brain will select PPO for our training session. We're good to continue with this algorithm. Thank you. See you once the training has finished.
