Hello. I'm Juan Vergara. Welcome back. We let our training session continue but we notice now that training stopped fairly quickly. Our training session stopped at approximately 250,000 iterations. A yellow menu at the top indicates to continue training increase the no progress iteration limit training parameter By default. The limit is set to 250,000 iterations. Once the platform identifies that the brain has not learned for 250,000 consecutive iterations, it will stop training automatically for you. This mainly saves us computing resources when a brain has reached its peak of learning and no new champions have been found. We would like to know how much better and robust our brain could be by letting it run a bit longer. It is not a bad idea to increase it to double the number of iterations. Every 5 to 10 experiments. Even up to 750,000 iterations is not too much when done sporadically, it will allow you to evaluate the further learning your early stop training sessions could have had if there were no computing limitations whatsoever. Moving forward, Let's train all brains with a no progress situation limit of 500,000 iterations. The default value of 250,000 iterations is often too low for industrial use cases, click over the teach tab, then click over the stabilize concept. If not selected already. One of the sections on the right pane reads training parameters within this section, modify the no progress iteration limit parameter From 250,000 2 500,000. You'll see your train button is green again, let's click over it and resume training despite a low no progress iteration limit. We are surprised to see that our experiment didn't learn anything meaningful. This experiment failed to accomplish the objective. We talk to our simulation team and they verified for us that the initialization conditions are not too random. The ship starts up right with some velocity either vertically or horizontally. Moreover, they let us know that the control works as expected and that no randomness is introduced in the computation of next states. The sim seems to not be the problem given the scenarios are not too hard and that the simulation works as expected. We decide to investigate the states and actions for any possible simplifications. Among the eight states exposed to the brain, we identified two that are possibly unnecessary left leg and right leg. Since stabilize is a concept where landing is not planned, we do not require those variables as the states for the concept removing them from the state space could infer quicker training. Nonetheless, we do not expect these to have a significant effect since doing most of the training, these values are going to be zero Among the three actions provided to the brain identify one unneeded dimensionality, the right and the left thrusts act perpendicularly to one another. After talking with the subject matter experts we clarify that activating the left and right engine at the same time and with the same strength would be a waste of fuel, they would cancel each other without influencing in the spaceship movement nor position This small detail could be highly affecting brain training. Thus we decided to simplify the action space to help the brain learn a sensible policy. We modify the control actions for left and right engine by combining them into a single control from minus 1 to 0. We activate the left thrust while from 0 to 1, we activate the right thrust. We talked with our simulation engineers and they pushed a new version of the simulation to observe. This simulation has a new input variable engine2 when engine2 two is provisioned as an action along with engine1, engine left, an engine right are not needed. These two will be computed based on engine2. As discussed. Our hypothesis is that this simplification will deeply help brain training. Since we are first removing one action to be learned while also to removing one dimension of complexity. Where the brain might have been trying to explore applying both left and right thrust engines at the same time. Let's copy the brain version by right clicking on v1 and selecting the copy version. Command, then click over that new version to open it up. If you go to the train graph, you'll see that our training graph is empty copying versions, transfers the inkling file, but none of the learning This is true for modular brains as well. Move to the teach tab now and select the stabilize box. If not selected already. Remember to activate visual authoring. If you were off this configuration framework back to Visual editor, it's time to select the stabilize concept on the right pane on the type section. Hover over the pencil that contains the brain action struct, modifying engine left to become engine2 and adjust the ranges to go from throttleMin to throttleMax, then remove the engine right action. We are now good to save and close before we start training this version let's update the notes to be able to better track the changes we make, click over the notes tab and enter a new line before the existing text on the first line enter V01 plus engine two instead of engine left and right independently. The reason why we put it on the first line is to ensure that our first few words are visible on the left pane. You will now see no matter which version you're looking at that V02 is a remake of version one. This gets increasingly helpful when tracking 50-plus experiments. While it gives you the added benefit of having a backlog of all the changes you made to the existing experiment by just looking at the Notes tab. Stop we're good to start this training session already feel free to click on the green train button. Now note if you see your training session didn't start automatically, click over the train button again, select a simulation of your choice, your case, lunar lander and wait for a session to start note it is not only the simulation that you want to modify or rethink when performing experimentation with Bonsai. Bonsai training parameters can impact training too. We presented the impact of the no progress iteration limit already but there is another parameter worth discussing. Now the episode iteration limit. Let's go to our first brain version by clicking it on the left pane, go to the train tab if not selected already. The legend of the graph clearly displays the four objectives for this version. One avoid and three minimize objectives. There are conditions under which training and testing episodes will be cut short. For example, if your brain goals contain an avoid or drive objective our training and testing episodes are cut as soon as the defined conditions are not satisfied, you can look at the drive goal to further understand the way it operates and stops episodes early focusing on our training session. The avoid clause is hit whenever the ship crashes, which we expect to happen frequently at the very beginning of training. Note training sessions start with a set of randomized weights. The initial random brain that our session started with and which no other champion could beat, is an inefficient control. We can see that on our goal satisfaction graph, our brain is crashing 7% of the time 7.3 Actually, how many iterations of success is that we can actually click over the goal satisfaction button to show a drop down menu here, select episode iterations, hover over the graph to find out the mean episode length of our, evaluated episodes when training started By default, Bonsai runs for 1000 iterations, but that might be much longer than the number of iterations needed. The optimum episode length often comes from conversations with the subject matter experts where the deployment benchmark control is discussed in this case. Since we have already trained one experiment, we can look at the episode iteration graph to see how many iterations it's taking the brain to succeed at this task. In this case, we understand that the brain is not actually succeeding. It's failing by crashing the fact that episodes are failing at around 72 iterations. On average, across the testing episodes is already an indicative factor that the default value of 1000 iterations might be too much for our training purposes. The orders of magnitude are quite different. The benefit of having shorter episodes is giving the brain a clear indicator of success whenever it is first able to fully avoid crashing during any given episode. Moreover, at the very beginning of the training session, we are wasting computing power in trying to recover from scenarios where the ship might be already too unstable to recover. Our hypothesis for our next experiment is that by reducing the length of the episode, we will be able to reach 100% goal satisfaction faster. This helps save computational power invested in trying to recover from too unrealistic state spaces caused by brain exploration. We want episode iteration limit to be a reasonable value not too short so that you are giving the brain a chance of succeeding at the task when exposed to the dynamics of the system yet not too long. So that the brain doesn't waste training time in sub optimal complex or long trajectories that might arrive to the objective. Let's copy the second version of the brain to test the effect of reducing the value for the episode iteration limit. Right click over the version v02 this time and select copy version, click over this new version. If not selected automatically and go to the teach tab here, click over the stabilize concept again if not selected already and go to the training parameters again where we previously modified the no progress iteration limit. You will notice that no progress iteration limit parameter is still 500,000 iterations. We will be transferring that value every time we copy one version into a new one, modify the episode iteration limits. Now from 1000 to 150 iterations. We're just giving a bit of upper room over the actual average length of 70 iterations that our first brain was failing at time to jump into a note section, go to the notes tab and enter a new line before the entries existing up to this point On the first line, enter v02 plus Episode Iteration limit equals 150. You can start to see now how each change beautifully adds up from the previous version, giving exposure to the whole experimentation thread. Up to the current experiment being tested. We're good to start this third version of the brain by clicking over the train button. If your brain doesn't start automatically, click over the train button and select the simulation of your choice, in your case, lunar lander we will have to now wait for our three existing versions of the brain to fully finish training. When they all finish training, we will be able to compare performance across brains. Remember to avoid running too many experiments in parallel. It is one of the best pieces of advice that we can give you while we wait for training to finish, it's your turn, now go ahead and perform all the mentioned changes on this video. Up to this point, Then we can come back when your training sessions are finished so that you can compare your results two ours.