[SciPy-Dev] GSoC'21 participation SciPy

[Ralf Gommers]

On Mon, Feb 15, 2021 at 10:19 PM Robert Kern <robert.kern at gmail.com> wrote:

> On Mon, Feb 15, 2021 at 2:02 PM Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>>
>> On Mon, Feb 15, 2021 at 6:24 PM Pamphile Roy <roy.pamphile at gmail.com>
>> wrote:
>>
>>> *scipy.optimize:* Would it be wanted to have a possibility to have
>>> workers to evaluate the function during an optimization?
>>> In most industrial context, the function is not trivial and might
>>> require minutes if not hours or even days to compute.
>>> Having a simple way to first parallelise the runs would help. We have
>>> machines with easily ten cores now and it would be great to leverage this
>>> here.
>>>
>>
>> Definitely - see the mention of workers under
>> http://scipy.github.io/devdocs/roadmap.html#performance-improvements.
>>
>> Going that direction, having a more general infrastructure to handle
>>> external workers would be great.
>>>
>>
>> I'm assuming you mean something like standard multiprocessing, or using a
>> custom Pool object, for code that's trivially parallelizable. Both are
>> covered by the `workers` pattern. If you're thinking about something else,
>> can you elaborate?
>>
>
> A standard approach for this is to organize the implementation of the
> optimization algorithms in what's usually called an "ask-tell" interface.
> The minimize()-style interface is easy to implement from an ask-tell
> interface, but not vice-versa. Basically, you have the optimizer object
> expose two methods, ask(), which returns a next point to evaluate, and
> tell(), where you feed back the point and its evaluated function value.
> You're in charge of evaluating that function. This gives you a lot of
> flexibility in how to dispatch that function evaluation, and importantly,
> we don't have to commit to any dependencies! That's the user's job!
>
> scikit-optimize implements their optimizers in this style, for example.
> It's pretty common for optimizers that are geared towards expensive
> evaluations.
>
>   https://scikit-optimize.github.io/stable/auto_examples/ask-and-tell.html
>
> I think it might be a well-scoped GSoC project to start re-implementing a
> chosen handful of the algorithms in scipy.optimize in such an interface. It
> could even be a trial run as an external package (even in scikit-optimize,
> if they're amenable). Then we can evaluate whether we want to adopt that
> framework inside scipy.optimize and make a roadmap for re-implementing all
> of the algorithms in that style. It will be a technical challenge to adapt
> the FORTRAN-implemented algorithms to such an interface.
>
> I will not be available to mentor such a project, but that's the general
> approach that I would recommend. I think it would be a valuable addition.
>

Thanks Robert, that seems like an interesting exercise. This reminds me of
the "class based optimizers" proposal. That didn't mention ask-tell, but
the "reverse-communication" may be the same idea:
https://github.com/scipy/scipy/pull/8552
https://mail.python.org/pipermail/scipy-dev/2018-February/022449.html

Your comments and the scikit-optimize link are imho a better justification
for doing this exercise than we had before.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210216/3e702c96/attachment.html>