Okay, so we're talking about optimizing a nonlinear program, and we may see that that's a minimization problem with no constraints. So our analysis will be focusing on the objective function. Okay, that's coded f of alpha beta. Then we're going to do what? Well, we want to talk about convexity. So we talk about gradient and hashem. Right? So the gradient in hashen can be obtained in this and that way, so I don't want to do the complete deprivation here, because you should already be able to do it by yourself. If you differentiate your objective function by alpha, then you're going to get this one. Okay, so you may put these 2 here and you differentiate the whole thing here and you get a negative term. So that's why you get a negative term. If you are not too familiar with this, try to do it once by yourself, okay? For better, that's the same thing. So I'm going to skip it, and then you do your hashem. So, for example, you take this one and then differentiate this guy with alpha again. You're going to get 2 and why is that? Because alpha is here, this is a linear function of alpha. So, alpha, after your differentiation goes away, you have the negative number. Negative number, they cancel each other and you basically get submission of one. That's how you get. All right. If you do the differentiation of this with respect to beta, then pretty much is the same. It's just that now this is not one anymore, inside your submission, it is exciting. Okay, so that's how you get your heshin, or please try to help yourself by doing the derivations. And then we now show the objective function is convex. Why is that so we apply the rules regarding leading principle minus. So we want this and the whole thing on the determinant of the whole thing, both to be positive or non negative. So how that is true. So first, this is of course, true, right? This is positive. And then regarding the determinant of the whole thing, pretty much we are doing these times this minus that times that okay, we get this one. So all the things remaining from my point of view is high school arithmetics. But anyway, you get some things here. So the first thing is that once you multiply the whole thing here x1 plus x2, plus blah, blah, blah plus xn and multiply it by yourself. You're going to get several terms of xi square. That's how you get here from n to minus one. Okay, so that's one thing. And also you get several cross product terms. And after some careful eliminations, careful arrangements, we're going to see that all the things goes to the submission of several square terms. Okay, so that can be done. And once you do that, you immediately see, my God, this is so non negative. So the objective function is indeed convex. Awesome. Or alternatively, there is another way that maybe you will feel better. So let's say we define another function. So our f is the objective function is still this one. But now for each eye, we define fi okay, so for fi is just regarding data point xi. And then for xi you may simply take the square and get these six terms. All right, so with this now it may be, I hope it's very simple for everyone to do the differentiation and get the hashem magics. So now you're differentiation is respecting alpha, alpha, beta, beta. Okay, and then you're going to see that all the things goes to here and that simply tells us that this is a convex function, because very quickly you don't need to do some weird derivations or calculations. This is clearly on positive, semi definite. Okay, according to reading principal minus, So fi is convex for oi. Now the only thing remains is to show that the some mention of convex functions is also convex. So this is actually very simple. All right, so in general, this is true, and I'm going to skip the proof if you are interested, and if you don't know how to prove it, just type the sentence to whatever search engine you have. Somebody will tell you, or if you want to do it by yourself don't forget that we are talking about twice differentiable functions. Okay, so if you have a submission and you have several functions here, if you want to show the second order derivative of this function is some kind of positive or positive, semi-definite. Then you do it for everyone and then pretty much you're done. All right, so try to help yourself to do this, Okay? So in any way, we now are able to solve the problem because we have shown that this is indeed an unconstrained convex program. Okay, that carefully designed a simple linear regression model is an unconstrained, complex program. So now we have two options. Either we use numerical algorithms like gradient descent Newton's method, whatever to try to solve for the optimal alpha and beta. That can be done, but of course, because this is unconstrained, convex and actually, if you look at it carefully, it is crogetic. Okay, all of the objective function is the second order polynomial. So if we take a look at the first order condition, which is necessary and sufficient, then immediately we get to linear equations regarding alpha and the beta. Okay, that's something, you are optimal alpha and beta must satisfy. So this is your first order derivative with respect to alpha, this is respect to beta. They should get to zero and that's how you get two equalities for alpha and beta. Okay, so after several arrangements you're going to see that alpha is here beta is here. You get the coefficient for alpha coefficient for beta. I hope this is not too difficult for you to check by yourself. So you also have alpha here beta here coefficient. So, basically, now you just have a linear system, which is a two by two linear system, two variables to equalities, you know how to solve it, right? Goes through the elimination or what it means, whatever method you like, you actually can get the close form formula or analytical solution for alpha and beta. Now you know how to do it then once you do it, please try to do this by yourself and compare your result with your statistics has books or any online resource you have. You will see that, okay, so for the near regression, everybody used the formula How to get it. At least there's one way to solve the nonlinear program we just presented to you. Okay, so maybe for many of you, you have used computer programs, statistical software to calculate alpha and beta and see your regression reports whatever. Now you know why it can be solved so easily because you have a closed form formula
