[SciPy-Dev] Extending scipy.optimize.minimize with particle swarm

[Matt Newville]

Hi Andrew, Andrea,

On Fri, May 15, 2020 at 6:22 PM Andrew Nelson <andyfaff at gmail.com> wrote:

> Andrea's test function collection is super nice, and is able to produce
> good evidence for assessment in this area. I do have one minor disagreement
> with the way the benchmarks on
> http://infinity77.net/global_optimization/index.html were run, the
> maximum number of function evaluations is limited to 2 000.
>
> In any case, it's clear that implementations matter. For example,
> http://infinity77.net/global_optimization/multidimensional.html# would
> indicate that differential evolution isn't very good against the Ackley
> function, with zero successes. However, running the scipy implementation of
> differential-evolution indicates that it's actually quite good,
> https://gist.github.com/andyfaff/24c96a3d5dbc7b0272b2. The latter gist
> would approximate the output from running the optimize benchmarks.
>
>
I think that number of function evaluations does matter though, as that is
effectively runtime.  In your results (apparently a different
implementation of `differential_evolution` then Andrea used),
`differential_evolution` still took nearly 4000 function evaluations for
the Ackley function.  In Andrea's table, 4 of the other methods solved that
function more than 80% of the time in under 2000 function evaluations.
Ampgo was admittedly marginal for that particular function but got it ~40%
of the time in under 2000 evaluations. Naively, you could run that twice
and get it to ~64% success rate for 4000 function evaluations.

Anyway, using the results of such benchmarks in a consistent manner would
be a good way to decide which algorithms are the next ones to add.  That
seems to suggest that ampgo would be a higher priority than particle swarm.

As it turns out, Andrew was involved in getting the `ampgo` code ready for
lmfit (see https://github.com/lmfit/ampgo), though Renee Otten did most of
the work actually getting it to work with lmfit Parameters, and so on.
 Still, that code from Andrea and Andrew is MIT licensed and could be put
into scipy.

Cheers,

--Matt Newville
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200516/93840905/attachment-0001.html>