So we were talking about instances where the design matrix X was orthonormal, it had orthonormal columns, so that X transpose x was the identity matrix. And let's talk about some of the most famous cases where this happens. Easily the biggest one is the so called Fourier basis. So in the Fourier basis, the columns of X are trigonometric. So imagine if your y is a time series. And then the columns of x are trigonometric terms, basically sines and cosines of different periods, from slow, low varying signals to really high-frequency signals toward the end of the x matrix. So what this means is that the coefficients associated from a least squares fit, from these basis elements and my observed time series y, the coefficients are just the inner product of these specific columns. Let's say xi is column i from x, and y. And so the transformation that takes y to the collection of these basis elements, in other words, that takes y to x transpose y, that transformation is called the Fourier transformation, or the discrete Fourier transformation. In fact, it can be done faster then this calculation. It can be done capitalizing on some of the redundancies in trigonometric relationships. It can be done faster to result in the so-called fast discrete Fourier transform. What you get with the Fourier transform is, each coefficient is the amount of sort of the signal that's correlated with a sine or a cosine turn of that frequency. So as an example, if I had a signal that looked like this. So it looks like maybe a sine term and a cosine term of, a slow variation sine term and a high-frequency term added together, then those two terms, the coefficients associated with those two terms, will wind up having big coefficients when we do x transpose y. So when we want to reconstruct the signal again, that's just summation, right, the basis elements times the inner product of the x's and the y's. So we can do tricks, for example, if we want to reconstruct the signal, but only omit, only include those terms that are very low frequency. So for example, if we want this big rolling term but we want to get rid of the high frequency information, we might do what is called a low pass filter. So we would let the low-frequency terms through, and we would filter out the high-frequency terms. So this would amount to, just in our reconstructed signal here, only adding those components associated with the low-frequency terms in the basis. And conversely, we might want to just capture the high-frequency stuff and filter out the low-frequency stuff. So in this sum, we would just take the high-frequency terms and omit the others. And then the reconstructed signal would have filtered out the low-frequency information. So again, this all boils down to least squares, like we've been studying. However, I think it's fair to say that signal processors tend to think of this in a different way. But it's nice to put this very important concept, Fourier analysis, in the domain of least squares, which we've been talking about in this class. The second basis that I would mention to discuss a little bit is so-called wavelet bases. And wavelet bases are similar to Fourier bases. They're an orthonormal basis that have some nice properties associated with them. And there is a discrete wavelet transform, just like there's a discrete Fourier transform. But it's actually able to get the transform faster than if you were to do it by calculating each of these inner products by themselves. Again, the Fourier basis and the wavelet basis, though, to really discuss them we'd have to get into the nitty-gritty of the construction of the bases. So I think I'm going to only then cover, really, in a little bit more detail, only one of the three cases that's a little bit more statistical, one of the three most important cases of orthonormal bases that's a little bit more statistical. And that's so-called principal component bases. So in the next lecture, we're going to talk about a particularly important version of a basis where x transpose x = I.
