[SciPy-Dev] GSoC'21 participation SciPy

[Robert Kern]

On Tue, Feb 16, 2021 at 9:03 AM Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
> On Mon, Feb 15, 2021 at 10:19 PM Robert Kern <robert.kern at gmail.com>
> wrote:
>
>> On Mon, Feb 15, 2021 at 2:02 PM Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>> On Mon, Feb 15, 2021 at 6:24 PM Pamphile Roy <roy.pamphile at gmail.com>
>>> wrote:
>>>
>>>> *scipy.optimize:* Would it be wanted to have a possibility to have
>>>> workers to evaluate the function during an optimization?
>>>> In most industrial context, the function is not trivial and might
>>>> require minutes if not hours or even days to compute.
>>>> Having a simple way to first parallelise the runs would help. We have
>>>> machines with easily ten cores now and it would be great to leverage this
>>>> here.
>>>>
>>>
>>> Definitely - see the mention of workers under
>>> http://scipy.github.io/devdocs/roadmap.html#performance-improvements.
>>>
>>> Going that direction, having a more general infrastructure to handle
>>>> external workers would be great.
>>>>
>>>
>>> I'm assuming you mean something like standard multiprocessing, or using
>>> a custom Pool object, for code that's trivially parallelizable. Both are
>>> covered by the `workers` pattern. If you're thinking about something else,
>>> can you elaborate?
>>>
>>
>> A standard approach for this is to organize the implementation of the
>> optimization algorithms in what's usually called an "ask-tell" interface.
>> The minimize()-style interface is easy to implement from an ask-tell
>> interface, but not vice-versa. Basically, you have the optimizer object
>> expose two methods, ask(), which returns a next point to evaluate, and
>> tell(), where you feed back the point and its evaluated function value.
>> You're in charge of evaluating that function. This gives you a lot of
>> flexibility in how to dispatch that function evaluation, and importantly,
>> we don't have to commit to any dependencies! That's the user's job!
>>
>> scikit-optimize implements their optimizers in this style, for example.
>> It's pretty common for optimizers that are geared towards expensive
>> evaluations.
>>
>>
>> https://scikit-optimize.github.io/stable/auto_examples/ask-and-tell.html
>>
>> I think it might be a well-scoped GSoC project to start re-implementing a
>> chosen handful of the algorithms in scipy.optimize in such an interface. It
>> could even be a trial run as an external package (even in scikit-optimize,
>> if they're amenable). Then we can evaluate whether we want to adopt that
>> framework inside scipy.optimize and make a roadmap for re-implementing all
>> of the algorithms in that style. It will be a technical challenge to adapt
>> the FORTRAN-implemented algorithms to such an interface.
>>
>> I will not be available to mentor such a project, but that's the general
>> approach that I would recommend. I think it would be a valuable addition.
>>
>
> Thanks Robert, that seems like an interesting exercise. This reminds me of
> the "class based optimizers" proposal. That didn't mention ask-tell, but
> the "reverse-communication" may be the same idea:
> https://github.com/scipy/scipy/pull/8552
> https://mail.python.org/pipermail/scipy-dev/2018-February/022449.html
>
> Your comments and the scikit-optimize link are imho a better justification
> for doing this exercise than we had before.
>

Yes, "reverse communication" is a FORTRAN-era term for the same general
idea. In FORTRAN reverse communication APIs, you would generally call the
one optimizer subroutine over and over again, passing in the current state
and function evaluation and reading the next point to evaluate (and other
state) from "intent out" variables. "ask-tell" is a somewhat more specific
instance of that idea, just in an OO context for which it's just a fairly
obvious design pattern once you have chosen to go OO and free yourself from
the constraints of a FORTRAN subroutine.

-- 
Robert Kern
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210217/ad7ec07a/attachment-0001.html>