[scikit-learn] scikit-learn Digest, Vol 43, Issue 24
Mike Smith
javaeurusd at gmail.com
Sat Oct 12 17:04:12 EDT 2019
"... > If I should expect good results on a pc, scikit says that needing
gpu power is
> obsolete, since certain scikit models perform better (than ml designed
for gpu)
> that are not designed for gpu, for that reason. Is this true?"
Where do you see this written? I think that you are looking for overly
simple stories that you are not true."
Gael, see the below from the scikit-learn FAQ. You can also find this
yourself at the main FAQ:
[image: 2019-10-12 14_00_05-Frequently Asked Questions — scikit-learn
0.21.3 documentation.png]
On Sat, Oct 12, 2019 at 9:03 AM <scikit-learn-request at python.org> wrote:
> Send scikit-learn mailing list submissions to
> scikit-learn at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman/listinfo/scikit-learn
> or, via email, send a message with subject or body 'help' to
> scikit-learn-request at python.org
>
> You can reach the person managing the list at
> scikit-learn-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of scikit-learn digest..."
>
>
> Today's Topics:
>
> 1. Re: Is scikit-learn implying neural nets are the best
> regressor? (Gael Varoquaux)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 11 Oct 2019 13:34:33 -0400
> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> To: Scikit-learn mailing list <scikit-learn at python.org>
> Subject: Re: [scikit-learn] Is scikit-learn implying neural nets are
> the best regressor?
> Message-ID: <20191011173433.bbywiqnwjjpvsi4r at phare.normalesup.org>
> Content-Type: text/plain; charset=iso-8859-1
>
> On Fri, Oct 11, 2019 at 10:10:32AM -0700, Mike Smith wrote:
> > In other words, according to that arrangement, is scikit-learn implying
> that
> > section 1.17 is the best regressor out of the listed, 1.1 to 1.17?
>
> No.
>
> First they are not ordered in order of complexity (Naive Bayes is
> arguably simpler than Gaussian Processes). Second complexity does not
> imply better prediction.
>
> > If I should expect good results on a pc, scikit says that needing gpu
> power is
> > obsolete, since certain scikit models perform better (than ml designed
> for gpu)
> > that are not designed for gpu, for that reason. Is this true?
>
> Where do you see this written? I think that you are looking for overly
> simple stories that you are not true.
>
> > How much hardware is a practical expectation for running the best
> > scikit models and getting the best results?
>
> This is too vague a question for which there is no answer.
>
> Ga?l
>
> > On Fri, Oct 11, 2019 at 9:02 AM <scikit-learn-request at python.org> wrote:
>
> > Send scikit-learn mailing list submissions to
> > ? ? ? ? scikit-learn at python.org
>
> > To subscribe or unsubscribe via the World Wide Web, visit
> > ? ? ? ? https://mail.python.org/mailman/listinfo/scikit-learn
> > or, via email, send a message with subject or body 'help' to
> > ? ? ? ? scikit-learn-request at python.org
>
> > You can reach the person managing the list at
> > ? ? ? ? scikit-learn-owner at python.org
>
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of scikit-learn digest..."
>
>
> > Today's Topics:
>
> > ? ?1. Re: logistic regression results are not stable between
> > ? ? ? solvers (Andreas Mueller)
>
>
> >
> ----------------------------------------------------------------------
>
> > Message: 1
> > Date: Fri, 11 Oct 2019 15:42:58 +0200
> > From: Andreas Mueller <t3kcit at gmail.com>
> > To: scikit-learn at python.org
> > Subject: Re: [scikit-learn] logistic regression results are not
> stable
> > ? ? ? ? between solvers
> > Message-ID: <d55949d6-3355-f892-f6b3-030edf1c7947 at gmail.com>
> > Content-Type: text/plain; charset="utf-8"; Format="flowed"
>
>
>
> > On 10/10/19 1:14 PM, Beno?t Presles wrote:
>
> > > Thanks for your answers.
>
> > > On my real data, I do not have so many samples. I have a bit more
> than
> > > 200 samples in total and I also would like to get some results with
> > > unpenalized logisitic regression.
> > > What do you suggest? Should I switch to the lbfgs solver?
> > Yes.
> > > Am I sure that with this solver I will not have any convergence
> issue
> > > and always get the good result? Indeed, I did not get any
> convergence
> > > warning with saga, so I thought everything was fine. I noticed some
> > > issues only when I decided to test several solvers. Without
> comparing
> > > the results across solvers, how to be sure that the optimisation
> goes
> > > well? Shouldn't scikit-learn warn the user somehow if it is not
> the case?
> > We should attempt to warn in the SAGA solver if it doesn't converge.
> > That it doesn't raise a convergence warning should probably be
> > considered a bug.
> > It uses the maximum weight change as a stopping criterion right now.
> > We could probably compute the dual objective once in the end to see
> if
> > we converged, right? Or is that not possible with SAGA? If not, we
> might
> > want to caution that no convergence warning will be raised.
>
>
> > > At last, I was using saga because I also wanted to do some feature
> > > selection by using l1 penalty which is not supported by lbfgs...
> > You can use liblinear then.
>
>
>
> > > Best regards,
> > > Ben
>
>
> > > Le 09/10/2019 ? 23:39, Guillaume Lema?tre a ?crit?:
> > >> Ups I did not see the answer of Roman. Sorry about that. It is
> coming
> > >> back to the same conclusion :)
>
> > >> On Wed, 9 Oct 2019 at 23:37, Guillaume Lema?tre
> > >> <g.lemaitre58 at gmail.com <mailto:g.lemaitre58 at gmail.com>> wrote:
>
> > >>? ? ?Uhm actually increasing to 10000 samples solve the convergence
> > issue.
> > >>? ? ?SAGA is not designed to work with a so small sample size most
> > >>? ? ?probably.
>
> > >>? ? ?On Wed, 9 Oct 2019 at 23:36, Guillaume Lema?tre
> > >>? ? ?<g.lemaitre58 at gmail.com <mailto:g.lemaitre58 at gmail.com>>
> wrote:
>
> > >>? ? ? ? ?I slightly change the bench such that it uses pipeline and
> > >>? ? ? ? ?plotted the coefficient:
>
> > >>? ? ? ? ?https://gist.github.com/glemaitre/
> > 8fcc24bdfc7dc38ca0c09c56e26b9386
>
> > >>? ? ? ? ?I only see one of the 10 splits where SAGA is not
> converging,
> > >>? ? ? ? ?otherwise the coefficients
> > >>? ? ? ? ?look very close (I don't attach the figure here but they
> can
> > >>? ? ? ? ?be plotted using the snippet).
> > >>? ? ? ? ?So apart from this second split, the other differences
> seems
> > >>? ? ? ? ?to be numerical instability.
>
> > >>? ? ? ? ?Where I have some concern is regarding the convergence
> rate
> > >>? ? ? ? ?of SAGA but I have no
> > >>? ? ? ? ?intuition to know if this is normal or not.
>
> > >>? ? ? ? ?On Wed, 9 Oct 2019 at 23:22, Roman Yurchak
> > >>? ? ? ? ?<rth.yurchak at gmail.com <mailto:rth.yurchak at gmail.com>>
> wrote:
>
> > >>? ? ? ? ? ? ?Ben,
>
> > >>? ? ? ? ? ? ?I can confirm your results with penalty='none' and
> C=1e9.
> > >>? ? ? ? ? ? ?In both cases,
> > >>? ? ? ? ? ? ?you are running a mostly unpenalized logisitic
> > >>? ? ? ? ? ? ?regression. Usually
> > >>? ? ? ? ? ? ?that's less numerically stable than with a small
> > >>? ? ? ? ? ? ?regularization,
> > >>? ? ? ? ? ? ?depending on the data collinearity.
>
> > >>? ? ? ? ? ? ?Running that same code with
> > >>? ? ? ? ? ? ?? - larger penalty ( smaller C values)
> > >>? ? ? ? ? ? ?? - or larger number of samples
> > >>? ? ? ? ? ? ?? yields for me the same coefficients (up to some
> > tolerance).
>
> > >>? ? ? ? ? ? ?You can also see that SAGA convergence is not good by
> the
> > >>? ? ? ? ? ? ?fact that it
> > >>? ? ? ? ? ? ?needs 196000 epochs/iterations to converge.
>
> > >>? ? ? ? ? ? ?Actually, I have often seen convergence issues with
> SAG
> > >>? ? ? ? ? ? ?on small
> > >>? ? ? ? ? ? ?datasets (in unit tests), not fully sure why.
>
> > >>? ? ? ? ? ? ?--
> > >>? ? ? ? ? ? ?Roman
>
> > >>? ? ? ? ? ? ?On 09/10/2019 22:10, serafim loukas wrote:
> > >>? ? ? ? ? ? ?> The predictions across solver are exactly the same
> when
> > >>? ? ? ? ? ? ?I run the code.
> > >>? ? ? ? ? ? ?> I am using 0.21.3 version. What is yours?
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?> In [13]: import sklearn
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?> In [14]: sklearn.__version__
> > >>? ? ? ? ? ? ?> Out[14]: '0.21.3'
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?> Serafeim
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?>> On 9 Oct 2019, at 21:44, Beno?t Presles
> > >>? ? ? ? ? ? ?<benoit.presles at u-bourgogne.fr
> > >>? ? ? ? ? ? ?<mailto:benoit.presles at u-bourgogne.fr>
> > >>? ? ? ? ? ? ?>> <mailto:benoit.presles at u-bourgogne.fr
> > >>? ? ? ? ? ? ?<mailto:benoit.presles at u-bourgogne.fr>>> wrote:
> > >>? ? ? ? ? ? ?>>
> > >>? ? ? ? ? ? ?>> (y_pred_lbfgs==y_pred_saga).all() == False
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?>
> > >>? ? ? ? ? ? ?> _______________________________________________
> > >>? ? ? ? ? ? ?> scikit-learn mailing list
> > >>? ? ? ? ? ? ?> scikit-learn at python.org <mailto:
> scikit-learn at python.org>
> > >>? ? ? ? ? ? ?>
> https://mail.python.org/mailman/listinfo/scikit-learn
> > >>? ? ? ? ? ? ?>
>
> > >>? ? ? ? ? ? ?_______________________________________________
> > >>? ? ? ? ? ? ?scikit-learn mailing list
> > >>? ? ? ? ? ? ?scikit-learn at python.org <mailto:
> scikit-learn at python.org>
> > >>? ? ? ? ? ? ?https://mail.python.org/mailman/listinfo/scikit-learn
>
>
>
> > >>? ? ? ? ?--
> > >>? ? ? ? ?Guillaume Lemaitre
> > >>? ? ? ? ?Scikit-learn @ Inria Foundation
> > >>? ? ? ? ?https://glemaitre.github.io/
>
>
>
> > >>? ? ?--
> > >>? ? ?Guillaume Lemaitre
> > >>? ? ?Scikit-learn @ Inria Foundation
> > >>? ? ?https://glemaitre.github.io/
>
>
>
> > >> --
> > >> Guillaume Lemaitre
> > >> Scikit-learn @ Inria Foundation
> > >> https://glemaitre.github.io/
>
> > >> _______________________________________________
> > >> scikit-learn mailing list
> > >> scikit-learn at python.org
> > >> https://mail.python.org/mailman/listinfo/scikit-learn
>
> > > _______________________________________________
> > > scikit-learn mailing list
> > > scikit-learn at python.org
> > > https://mail.python.org/mailman/listinfo/scikit-learn
>
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> http://mail.python.org/pipermail/scikit-learn/attachments/20191011/
> > a7052cd9/attachment-0001.html>
>
> > ------------------------------
>
> > Subject: Digest Footer
>
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
>
>
> > ------------------------------
>
> > End of scikit-learn Digest, Vol 43, Issue 21
> > ********************************************
>
>
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
>
>
> --
> Gael Varoquaux
> Research Director, INRIA Visiting professor, McGill
> http://gael-varoquaux.info http://twitter.com/GaelVaroquaux
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
> ------------------------------
>
> End of scikit-learn Digest, Vol 43, Issue 24
> ********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20191012/6959d075/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2019-10-12 14_00_05-Frequently Asked Questions ? scikit-learn 0.21.3 documentation.png
Type: image/png
Size: 26245 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20191012/6959d075/attachment-0001.png>
More information about the scikit-learn
mailing list