[scikit-learn] Why ridge regression can solve multicollinearity?

[josef.pktd at gmail.com]

On Wed, Jan 8, 2020 at 9:38 PM lampahome <pahome.chen at mirlab.org> wrote:

>
>
> Stuart Reynolds <stuart at stuartreynolds.net> 於 2020年1月9日 週四 上午10:33寫道：
>
>> Correlated features typically have the property that they are tending to
>> be similarly predictive of the outcome.
>>
>> L1 and L2 are both a preference for low coefficients.
>> If a coefficient can be reduced yet another coefficient maintains similar
>> loss, the these regularization methods prefer this solution.
>> If you use L1 or L2, you should mean and variance normalize your features.
>>
>>
> You mean LASSO and RIDGE both solve multilinearity?
>

LASSO has the reputation not to be good when there is multicollinearity,
that's why elastic net L1 + L2 was introduced, AFAIK

With multicollinearity the length of the parameter vector, beta' beta, is
too large and L2, Ridge shrinks it.

Josef



>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20200108/b959d061/attachment.html>