The most important step in building a linear regression model is to construct a cost function, as the cost function tells us how well the model is done, so that we can try to fit the parameters of the model better. As one of the most common and important concepts in machine learning, cost function is used for linear regression and training many advanced Al models in the world.
When fitting a univariate regression equation, we need to solve for the values of the two parameters w and b, and we can use the cost function to determine whether (w, b) is optimal or the best case:
At this point, we can use the cost function to:
when meeting ,the optimal (w, b) solution can be obtained.
If it is one parameter, the cost function can generally be intuitively seen through a two-dimensional curve. If there are two parameters, the cost function can be seen from the three-dimensional image, and the more parameters, the more complex it becomes.
When there are 2 parameters, the cost function is a three-dimensional image. As shown in the following figure:
Among them, the height is the value of the cost function. At this point, we can take the solution of (w, b) on the graph and fit the first-order regression equation to observe the degree of error of the corresponding first-order equation.
Of course, as a cost function used to fit regression equations, it has limited applicability to training algorithms. Therefore, to address this issue, we have also proposed a more efficient training method, namely gradient descent. In the next section, we will bring you the relevant knowledge of gradient descent.