[MUSIC] Next we want to cover all the AI approaches and how the modern AI came to be. And if we traced through this obligatory history lesson, you see that since the 1950 AI or more specifically, symbolic AI work has always been there. At that time, it was more logical programming the engineer, him or her is responsible for putting the logic together, how AI should behave. And overall, the compute wasn't quite there the data wasn't quite there, but it was largely successful in terms of the imagination of what is possible with AI, and it's something that we should work on the society. And then came a little bit of a secondary revival in the sense of the machine learning in the 1980s, where the success really was still based on the human or the engineer, or the data scientist who was looking at all this data and figuring out what is the variable. What is the feature that is important and that was called feature engineering, where it was still the human that decided. Okay, how do I work with this data and then tell the computer or the model what to look for and this is the weight, and this is how do you put it together? So it was up to the humans still. And then, in typical presentations of deep learning, you'll see that most people talk about how modern AI or deep learning, is just a small subgroup of machine learning. While that is true in terms of how the structure and how we will put that in, it would be, I think, a little bit of a mistake to understand put it into the proper context of how revolutionary it is, right? It's kind of saying Assembly language and C programming it's the same if we really come from programming background, for example. And really, it's the advent of compiler and the high level, object oriented programming and these highly productive sort of modern languages that allows it. And the responsibility falls on the compiler to make sure the assembly of the machine language captures the same behavior and it runs functionally the same and correct. So same thing here with deep learning in terms of what I see is that we only give the model of the data and it ingested and figure out how to structure it together. There is no longer the engineer, the scientists who is responsible, figuring out the feature or programming it manually. But it's the model itself that learn what is important and what is seen. Now the danger there is the data must be representative and correct. But the nice thing is, if you get the data right, the model will based on this tracking that we're doing with the loss function and all based on this accuracy, we'll see how well it captures the data that really it depends on the data that we give it. That's why and it shouldn't be surprising the very first kind of network and models that tends to work well with supervised learning and it's still the most prevalent today. And in terms of this approach, it usually is separated into two different phases. First one is you have all of this data, and it has to be labeled because it's supervised right? You're literally watching over the model as it learned, and you're feeding it. And in order for it to learn your label, that each piece of information as this is human, this is a chair, etc. So that all the example it's seeing there's a data label connected to it, telling it exactly what that is. And we usually also call it the ground truth is something that we know for ourselves that we embedded in the data and the labels, and the task of the model is to learn the function that fits the data. So that's the short and quick of how supervised learning is and how well it learns it. We track those metrics, but in general, the accuracy rate we know if it learned the data or not. But really, the model's job is just to based on the data that it's been given, how well it fits to the data. And then inferences where you take that model and then you run it on new data or new stimuli that it hasn't seen before. And based on that particular function that it learned it's going to do the prediction so it doesn't know how well that prediction will work it just know based on the function that it has, whatever the data will be. So that's why, in supervised learning, sometimes when the model fails, it fails in incomprehensive and spectacular ways, because maybe just a small feature that you didn't really think was important. But somehow, based on what it thought was important, it learned it right? One of the canonical examples that we give is that wolf detection kind of a thing that people give it. That was unexpected bias introduced in the data set, where it's always against the snowy background. So the model actually didn't learn about the wolf and detection of the wolf it kind of learned it in the context of the snow background. So if you just give a wolf but in a different background, the model will be completely off. So, as I said, this is the most successful in the 2010s because for the first time, we got the requisite massive amount of data from the Internet and things like that. So because all the amount of data is available and it's labeled, we were able to collect that repository and it was able to learn and go through lots of lots of data to get so many examples that it starts to learn really well. But in order to do that and crunch all that data, you need to compute the storage and everything and somehow with Moore's law with the era that we live in, that was also created. So we had this convergence of those two things, so we also cannot call it, in a way, some brute force AI, right? There's nothing, in a way, intelligent about it, but because of the data and the computer, it's able to have much higher accuracy he's probably better than most human in these narrow tasks. Now, and it's been produced and deployed in two particular domains computer vision and natural language processing these days. Next is unsupervised learning and, unlike supervised learning, instead of giving it the label and the data, you only give it data, and it's up to the model to figure out how to represent the data. And that's why we need high dimensionality, because you need to be able to slice it the data into different categories, right? And if you only do it in a 2D plane, the freedom and how much you can separate the data is limited. But if you have high dimensionality, what it means to be neighbors and how to group them together, there's a lot more degrees of freedom, and that's how you are able to identify and group this data in many different ways. So that's exactly what an unsupervised learning model will do based on all the data it's given, it's going to learn to categorize it, and then it's going to come up with the category itself, potentially right? So none of it is labeled, it's just learning how to organize the data on its own. That's why it's unsupervised learning. So these are the two main approaches that we have in machine learning these days, especially with modern AI supervised and unsupervised learning. All right, so then we can go on to the quiz about GANs. So in GANs, as we've said before, there's a generator portion, there's a discriminated fortune and when you put it together is a generative, adversarial network. So for this task, we hope that you can figure out is generator supervised, unsupervised or none of the above or combination. Same thing with the discriminator and the whole generative adversarial network to see how well you can separate the two supervised versus unsupervised,. The third type of modern AI approaches called reinforcement learning. And it's also a fairly old idea in terms of academic research. And just like the name implies reinforcement learning is where you're learning to make a decision. And as we go through in normal life, whenever we do something right, we get some sort of award. And whenever we do something wrong, we get some sort of punishment, right? That's a very crude, simplistic view of the world. In general imagine if you had an objective function that can calculate that award for a given decision. Then reinforcement learning is basically trying to get as high of a score or optimizing the reward function and getting it making these decisions that are making it as big as possible. This was started since the 1980s this formed the basis for a modern approach, which combined ideas from reinforcement learning and combine it with deep learning to create deep reinforcement learning. So in this case, the agent itself is a deep neural network. So whoever is making the decision is model as a deep neural network, as we have seen in all the previous examples. Because the reward function closely resembles the score, having a gaming environment where the school is only president is being updated all the time. It makes it a very fertile ground for a deep reinforcement learning agents to learn. And that's exactly what has happened in 2013 with DeepMind's Atari games. They created this network and infrastructure for these networks to learn in the gaming context. But these networks obviously learned very, very narrow tasks of only playing a particular kind of games. But the kind of games that they are able to learn, the complexity goes higher and higher, right? And we thought that chess or go those type of games with exponential increase in the search space that has to go through that it wasn't possible for these neural networks to learn, but that proved to be not true with 2015 AlphaGo kind of thing. Whereas in popular culture a lot of people know about AlphaGo that may think it's either supervised learning, unsupervised learning. But it's actually this third type of approach with deep reinforcement learning. And then, if the agent is better at being human, what's better than one agent? Well, two agents playing against each other and discovering all of this unexplored strategies and things that humans wouldn't think to do. And that's exactly what AlphaZero was in 2017, so you can see that progression of technology is always step by stuff like this. Lastly, the fourth type of active learning approaches is a very active research area because what you really want is to reduce the dependency on the amount of data for these models to learn. So in active learning, you target it, and the way you target it is the model itself. Identify the lowest confidence area, and then it will ask for more data specifically for that uncertain area, right? Just like did you think of anybody who is learning something that I know? Well, I don't need more example of, but something I'm unsure about or at the edge of the boundary I can ask more data on that, and that's exactly what active learning is. And then it in turns uses that new data to update the model for higher confidence. These are the natural things that you would expect the research to happen in this way. So we can guess where the future will be by just looking at what the current trends are. Currently there's bigger is better seems to be the theme. More data bigger neurons, bigger networks and more compute bigger machines, still giving incremental returns of people keep going bigger and better. But at some point there's going to be an S curve that we hit where it plateaus, and it's easy to see at those point what you want is more something like one shot learning. You see something once and then you know it kind of like how a child would learn instead of only understanding what something is after a million examples. That shouldn't be true intelligence in some sense and then having self supervised kind of way of learning overall. And then I personally think that it's going to be a big convergence between the two areas that we mentioned the training and inference where you train in isolation and then you deploy and inference something in the wild. I think those two should combine in such a way that they should be online continual learning loop for these future AI systems that maybe is growing with the subject of the person that it learned to be trained on. Those are clear future trends that it should continue in the future. Bigger is better, but I think also smarter. All right, so then with all the things that we worked on, I think the point here is that based on what you know about your own ethical framework, which of the four AI approaches would you think is best to implement? And that you would trust once it's been implemented, that the model really got your ethical framework correctly. So give that a shot. [MUSIC]
