[SciPy-Dev] WIP: Class based Optimizers

[Matthieu Brucher]

A long time ago, I wrote a class-based optimization framework. Not sure if
people actually care about class-based optimizers.
But if you want to create one, I would suggest you take a look at
scikits.optimization if classes is what you are looking for here.

Cheers,

Matthieu

2018-02-14 23:00 GMT+00:00 Scott Sievert <sievert.scott at gmail.com>:

> This would be the third optimizer API to the same solvers
>
> I agree. The ODE solvers were previously implemented with the iterative
> method, which was found to be less-than-ideal.
>
> What references are there on these? I’ve found [PR #6326] on the ODE
> solvers. What else is out there?
> Scott
>
> [PR #6326]: https://github.com/scipy/scipy/pull/6326
>
>
> On February 14, 2018 at 3:16:53 PM, xoviat (xoviat at gmail.com) wrote:
>
> Pauli,
>
>
>
> What do you think about returning the entire optimization history, similar
> to what is done with the ODE solvers, as I suggested?
>
>
>
> I think it would be necessary to make clearly a case what is actually
>
> gained with such reorganization. This would be the third optimizer API
>
> to the same solvers, and at first sight it's not clear to me if there
>
> is a strong need for it.
>
>
>
> I agree. The ODE solvers were previously implemented with the iterative
> method, which was found to be less-than-ideal.
>
>
>
> *From: *Pauli Virtanen <pav at iki.fi>
> *Sent: *Wednesday, February 14, 2018 3:12 PM
> *To: *scipy-dev at python.org
> *Subject: *Re: [SciPy-Dev] WIP: Class based Optimizers
>
>
>
> Hi,
>
>
>
> ke, 2018-02-14 kello 14:21 +1100, Andrew Nelson kirjoitti:
>
> > The #8414 PR on github (https://github.com/scipy/scipy/pull/8414)
>
> > introduces an `Optimize` class for `scipy.optimize`. The `Optimize`
>
> > class
>
> > can be advanced like an iterator, or be advanced to convergence with
>
> > the
>
> > `solve` method.
>
>
>
> I think it would be necessary to make clearly a case what is actually
>
> gained with such reorganization. This would be the third optimizer API
>
> to the same solvers, and at first sight it's not clear to me if there
>
> is a strong need for it.
>
>
>
> One point that one can argue is that
>
>
>
> solver = ...
>
> for foo in solver:
>
>    some_code
>
>
>
> is identical with regard to functionality to
>
>
>
> def callback(.., solver_info):
>
>     some_code
>
> solver(..., callback=callback)
>
>
>
> with the difference that the latter does not leak unnecessary private
>
> implementation details to the user code, requires only small API
>
> changes, and won't require extensive code reorganization.
>
>
>
> The use cases where one needs to continuously monitor convergence in
>
> minimization are also somewhat expert-oriented, and for most users I
>
> expect such API will not be as useful. (The situation with ODE solvers
>
> is I think somewhat different, as with them one e.g. often does not
>
> even know the time up to which to integrate a priori.)
>
>
>
> Writing class-based solvers requires rewriting code in a reverse-
>
> communication style, which inverts the structure of control flow. This
>
> can result to code that is more complex, and more difficult to
>
> maintain. There are some tricks one can do to retain more natural
>
> structure such as using generators/coroutines, but these are not always
>
> so nice to use in practice in Python in my experience, and also bring
>
> more complexity.
>
>
>
> The existing solvers in Scipy apart from L-BFGS-B are IIRC not written
>
> in a reverse-communication style, and converting them would be
>
> significant rewrite, particularly so for the solvers written in
>
> C/Fortran.
>
>
>
> The same applies also for tuning solver parameters during the
>
> iteration. The solvers AFAIK have not been designed with parameters
>
> that can be adjusted during run, and one would probably have to think
>
> carefully about each one (instead of e.g. exposing them as class
>
> attributes).
>
>
>
> So, I think there are several questions that one should consider here.
>
>
>
>                 Pauli
>
>
>
> _______________________________________________
>
> SciPy-Dev mailing list
>
> SciPy-Dev at python.org
>
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
>


-- 
Quantitative analyst, Ph.D.
Blog: http://blog.audio-tk.com/
LinkedIn: http://www.linkedin.com/in/matthieubrucher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180217/ff4386cb/attachment-0001.html>