Lasso

Last post of Regularization trilogy. Here we'll dive into "Elastic-Net Regularization". Often called the best of both worlds. Creates flexible, robust and interpretable solution. Jump in to see how combination of L1 and L2 penalty in loss function, work togther to make possible a great model.

While deriving the optimal solution for Lasso Regression via Coordinate Descent based optimization. Their comes a defining point where solution turns to Soft Thresholding Operation for calculating optimal 𝑤ⱼ, because sign(𝑤ⱼ) aligns with sign(z). This post helps with geometric intuition of how sign(𝑤ⱼ) aligns with sign(z) that enables Soft Thresholding Operation for calculating optimal 𝑤ⱼ.

I hope you liked Part 1 of this trilogy. Next we'll dive into "Lasso Regularization". The ruthless feature selection algorithm and a great simplifier. Prevent overfitting. Automatic feature selection by vanishing the coefficients of less important features. L1-penalty is the main reason for shrinkage. But how? It cannot be solved via Gradient Descent Algorithm needs a special optimization algorithm. Jump in to see which one?

A technique in ML that prevents overfitting in complex ML system, by adding a penalty to the Loss Function. Helps improve a model’s generalization ability. Introduced “Penalty” discourages the model from learning overly complex patterns in data, and instead learn the most visible pattern, hence later perform well on unseen data.