Articles

## Weighted Least Squares and locally weighted linear regression

From the post on Closed Form Solution for Linear regression, we computed the parameter vector  which minimizes the square of the error between the predicted value  and the actual output  for all  values in the training set. In that model all the  values in the training set is given equal importance.  Let us consider the case where it is known…

## Least Squares in Gaussian Noise – Maximum Likelihood

From the previous posts on Linear Regression (using Batch Gradient descent, Stochastic Gradient Descent, Closed form solution), we discussed couple of different ways to estimate the  parameter vector in the least square error sense for the given training set. However, how does the least square error criterion work when the training set is corrupted by…

## Closed form solution for linear regression

In the previous post on Batch Gradient Descent and Stochastic Gradient Descent, we looked at two iterative methods for finding the parameter vector  which minimizes the square of the error between the predicted value  and the actual output  for all  values in the training set. A closed form solution for finding the parameter vector  is possible, and in this post…