Regularized Linear Regression

$\hat{w}=({\mathbf{X}}^{{}^{\top }}\mathbf{X}+\lambda \mathbf{I}{)}^{-1}{\mathbf{X}}^{{}^{\top }}\mathbf{y}$

• The ideal weights/parameters of the linear model
• $\lambda$ is the regularization parameter

$\nabla J(\mathbf{w})=2({\mathbf{X}}^{{}^{\top }}\mathbf{Xw}-{\mathbf{X}}^{{}^{\top }}\mathbf{y}+\lambda \mathbf{w})$