[scikit-learn] L-BFGS in MLPClassifier

[Mathieu Blondel]

Thanks for this email. It is always nice to hear about success stories.

I assume the guilty party is Issam Laradji, as you can see from his Google
Summer of Code blog post:

http://issamlaradji.blogspot.jp/2014/06/week-3-gsoc-2014-extending-neural.html

L-BFGS is indeed usually a good default choice for medium-scale datasets.
It doesn't require any step size tuning and I found recently that it works
well for poorly conditioned problems.

You can also see a blog post by Nicolas Le Roux praising L-BFGS here:

http://labs.criteo.com/2014/09/poh-part-3-distributed-optimization/

Mathieu


On Sat, Aug 26, 2017 at 12:40 AM, Dr. Mario Michael Krell <
mario.michael.krell at gmail.com> wrote:

> To whoever programmed the MLPClassifier (with the L-BFGS solver),
>
> I just wanted to personally thank you and if I get your name(s), I would
> mention it/them in my paper additionally to the mandatory sklearn citation.
>
> I hope that sklearn will be keeping this algorithm forever in their
> library despite the increasing amount of established deep learning
> libraries that seem to make this code obsolete. For my small scale, more
> theoretic analysis, it worked much better than any other algorithm and I
> would not have gotten such surprising results. Due to the high quality
> implementation, the integration of a much better solver than SGD, and the
> respective good documentation, I could show empirically how the VC
> dimension and another property of MLPs (MacKay dimension) actually scale
> linear with the number of edges in the respective graph which helped us to
> provide a new much more strict upper bound (https://arxiv.org/abs/1708.
> 06019). This would have not been possible with other implementations. If
> there is an interest by the developers, I could try to contribute a
> tutorial documentation for sklearn. Just let me know.
>
> Thank you a lot!!!
>
> Best,
>
> Mario
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170826/6e9e886f/attachment.html>