Least-squares fitting

Consider a second-order polynomial, a nonlinear output of the network node, utilized for least-squares fitting of its input regressors  and  to

The error of approximation is given by

The minimal error is obtained for a that is a solution to the system of equations

where the summations are over all the instances.

The least-squares error can be evaluated on the validation set according to