[MUSIC] In this video, we'll conclude our discussion of neural networks by discussing a few applications of ANN to smart city domain. The first application that we discuss is the Land Use Identification and Prediction which were discussed in this paper titled “Using neural networks and GIS to forecast land use changes: A land transformation model.” In this, the authors study the land use evolution from 1970 to 1997 and then predict it for the future. The study area was Michigan's Grand Traverse Bay Watershed which was partitioned into 100m across 100m cells. The data was coded for each cell as one if the land use had changed and zero otherwise. For each cell there are 10 predictor variables of features such as agricultural density, distance to nearest highway, distance to nearest inland lake and so, on. The output is the likelihood of a non- urban cell changing to the urban. These are the results which were obtained for the year 1980, 90, 2000, 10 and 20. So, this it is 2000, 2010 and 2020 are the predictions into the future. Different cells are color coded to show the land uses type such as urban, semi-urban agriculture, suburban, forest and so, on. Now looking at a few pros and cons of applying neural network to these problems. Now looking at the pros. So, they have proven training method for multi layer nets which are able to learn any arbitrary function and are useful for learning nonlinear mapping. These methods works well with noisy data and generalize well given sufficient examples and have rapid recognitions speed. Cons include the model may get stuck in local minimum or it has high space and time complexity and a lengthy training time. Also, the decision making process is opaque. So, it's more like a black box. Over the years, there have been several advancements in ANN such as variations in feedforward architecture which include jump connection to output nodes, hidden nodes that vary in structure. We have also recurrent networks with feedback connections, probabilistic networks and unsupervised self-organizing networks. Now let's look at an example of python for multi layer perceptron MLP classifier. In this, first we import the module from sklearn.neural_network import MLPClassifier. We experiment on the wine dataset which is the chemical analysis of wines grown in the same region of Italy. But derived from three different cultivators. For training the model first we separate training and testing data using train test split and we pre-process the data in which we scale all the features. Using standard scaler. To train the model first we define the classifier as MLP is multi-layer perceptron classifier in which we say the hidden layer size is defined to be 13, 13, 13. That means it consists of three hidden layers. Each having 13 nodes. The maximum number of iteration is set to 500 for convergence. We then train the model using the dot fit function and the output is again an object consisting of these variables. The activation function which is relu, alpha and all other parameters. We predict and evaluate. So, the predictions are based on MLP.predict function and the confusion matrix as shown here. Class one is correctly labeled. That is 12 instances are correctly classified. For class two, out of 21 instances the model is able to correctly infer the label of 20 objects whereas one object was wrongly classified. Class three was again correctly classified for all instances. Now let's look at another problem which is common with neural networks, that is overfitting since neural network can model any arbitrary boundary, they tend to over fit the training data. To see this, let's create a data set where we have two features, one is “number of hours one slept” and “numbers of hours a student studied” and the dependent variable is the score on the test. So we randomly generate the data which is shown here. So, here for both the figures, the X axis is the number of hours sleeping and numbers of hours studying and the Y axis is the test score. As we can see this is a bit arbitrary. So, first we normalize the data and then we define the neural network NN as neural network then we train the network with this data. The figure shows the cost function during the train you can see as the number of iteration increases the training cost decreases sharply. And then saturate to a very low value. Then we test network for various combinations of sleep and study to see how the results vary. Again for the test data we normalize the data the same way the training data was normalized. And then we generate all of them into a single input matrix to see how the model performed on test data. To check for overfitting we divide the data into training and test set. So, we have train and test. And then we trained the network with the new training data. So, in this figure we can see that there are two curves the blue is curve the training data error and the orange one is the test data error. The training data error reduces sharply and then saturated to a low value but the test data error keeps on increasing and at the end becomes very high. That means the cost for test data is more than that of training data. So, this is a clear sign of overfitting. So, what can we do to reduce it? One possible solution is regularization in which we add a term to cost function that penalizes overly complex model. That is, we do not want very complex models and then we visualized costs during train. So, in this figure we can see the training error and the testing error are almost similar to each other. This means this model is not overfitting and it's perfect. In this video we discussed a few applications of ANN to smart cities generated data and talked about handling overfitting. Thank you. [MUSIC]
