In this lecture, I will introduce the first classification scheme that we're going to look at, which is nearest neighbor classification before we proceed to more complex alternatives in later lectures. So, essentially, the purpose of this lecture is really just to introduce the notation we'll be using for building classification algorithms later on before we move on to something more complex. Also, it demonstrates a non-learning solution to classification problems. So, most of the parts we have seen so far to classification or regression for that matter have, assume, we have some parameters of the model we are trying to fit. This is a much simpler approach that has no parameters whatsoever. Okay. So, this is what a classification problem might actually look like in practice. We have some data, we're looking to classify belonging to in the simplest case at least, one of two classes, those are positive and negative class. So this scatter plot represents some data I might have collected which has two features, and each data point has an associated classification. So, for example, this could be whatever you like, but maybe you have features representing the height and weight of different individuals, and your prediction you are trying to make is whether an individual is male or female. So those would be a two classes, one of which is positive or negative. But, it could be anything, really we just have data where we have a binary class, and we have two features we're trying to use to predict that class which we call positive and negative examples. Here, the examples are drawn as stars for negatives and I think they pep is for positives. But you can draw it however you like. In other words, we have some data which has two different classes and we'd like to have some function that give a new observation estimate, which of the two classes that point should belong to. So, what's the simplest algorithm we can come up with to classify a new data point? So, suppose we observe some point here denoted as X, and we'd like to estimate whether it's positive or negative, or we observe about those data points is those two-dimensional features associated with it. So, nearest neighbor is a very simple classification scheme we can use to solve this. It says, well, this data point probably has the same category or class as other data points with similar features. So to find the data point with the most similar features, let's just compute the distance of this data point or more precisely this data point's features all other data points. We'll find the nearest one, and the prediction we make says, this data point should have the same category or the same label as the nearest data point in our training set. So in this case, nearest point has negative label, and we will label this data point as negative. Okay. So let's be more precise about this, we're given a collection of data points X, that's really is a collection of feature vectors, is exactly the same as what we've been saying so far in our lectures on regression. You have labels y which are just true or false, and we see a new point z or z that we wish to label. Then classification should be done according to the following function which I'll try and break down. So first of all, what we're trying to predict is the label assigned to the new point z. What we're given to do that is the distance between z and all of our other data points, or really I mean the distance between Zth features and the features of each of the other data points. So this function here, is just computing the distance between z and each data point Xi. What we're still doing is finding the nearest point, so the data point Xi that minimizes this function. Finally, we're saying what is the label of that nearest point? So that's all that's going on in this somewhat complex mathematical formulation. Okay. So that's really it. That is nearest neighbor classification. Really what I wanted to introduce you to in this lecture is this notation we use classifiers, we still have this matrix of features X, and we still have labels y, the labels just happen to be binary classes.
