[SciPy-Dev] WIP: Class based Optimizers

[Scott Sievert]

This would be the third optimizer API to the same solvers

I agree. The ODE solvers were previously implemented with the iterative
method, which was found to be less-than-ideal.

What references are there on these? I’ve found [PR #6326] on the ODE
solvers. What else is out there?
Scott

[PR #6326]: https://github.com/scipy/scipy/pull/6326


On February 14, 2018 at 3:16:53 PM, xoviat (xoviat at gmail.com) wrote:

Pauli,



What do you think about returning the entire optimization history, similar
to what is done with the ODE solvers, as I suggested?



I think it would be necessary to make clearly a case what is actually

gained with such reorganization. This would be the third optimizer API

to the same solvers, and at first sight it's not clear to me if there

is a strong need for it.



I agree. The ODE solvers were previously implemented with the iterative
method, which was found to be less-than-ideal.



*From: *Pauli Virtanen <pav at iki.fi>
*Sent: *Wednesday, February 14, 2018 3:12 PM
*To: *scipy-dev at python.org
*Subject: *Re: [SciPy-Dev] WIP: Class based Optimizers



Hi,



ke, 2018-02-14 kello 14:21 +1100, Andrew Nelson kirjoitti:

> The #8414 PR on github (https://github.com/scipy/scipy/pull/8414)

> introduces an `Optimize` class for `scipy.optimize`. The `Optimize`

> class

> can be advanced like an iterator, or be advanced to convergence with

> the

> `solve` method.



I think it would be necessary to make clearly a case what is actually

gained with such reorganization. This would be the third optimizer API

to the same solvers, and at first sight it's not clear to me if there

is a strong need for it.



One point that one can argue is that



solver = ...

for foo in solver:

   some_code



is identical with regard to functionality to



def callback(.., solver_info):

    some_code

solver(..., callback=callback)



with the difference that the latter does not leak unnecessary private

implementation details to the user code, requires only small API

changes, and won't require extensive code reorganization.



The use cases where one needs to continuously monitor convergence in

minimization are also somewhat expert-oriented, and for most users I

expect such API will not be as useful. (The situation with ODE solvers

is I think somewhat different, as with them one e.g. often does not

even know the time up to which to integrate a priori.)



Writing class-based solvers requires rewriting code in a reverse-

communication style, which inverts the structure of control flow. This

can result to code that is more complex, and more difficult to

maintain. There are some tricks one can do to retain more natural

structure such as using generators/coroutines, but these are not always

so nice to use in practice in Python in my experience, and also bring

more complexity.



The existing solvers in Scipy apart from L-BFGS-B are IIRC not written

in a reverse-communication style, and converting them would be

significant rewrite, particularly so for the solvers written in

C/Fortran.



The same applies also for tuning solver parameters during the

iteration. The solvers AFAIK have not been designed with parameters

that can be adjusted during run, and one would probably have to think

carefully about each one (instead of e.g. exposing them as class

attributes).



So, I think there are several questions that one should consider here.



                Pauli



_______________________________________________

SciPy-Dev mailing list

SciPy-Dev at python.org

https://mail.python.org/mailman/listinfo/scipy-dev


_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at python.org
https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180214/4b37746d/attachment-0001.html>