[SciPy-Dev] Extending scipy.optimize.minimize with particle swarm

[Andrew Nelson]

Andrea's test function collection is super nice, and is able to produce
good evidence for assessment in this area. I do have one minor disagreement
with the way the benchmarks on
http://infinity77.net/global_optimization/index.html were run, the maximum
number of function evaluations is limited to 2 000.

In any case, it's clear that implementations matter. For example,
http://infinity77.net/global_optimization/multidimensional.html# would
indicate that differential evolution isn't very good against the Ackley
function, with zero successes. However, running the scipy implementation of
differential-evolution indicates that it's actually quite good,
https://gist.github.com/andyfaff/24c96a3d5dbc7b0272b2. The latter gist
would approximate the output from running the optimize benchmarks.

(Note that the gist is quite old, and the `anneal` column is from the old
anneal function that was removed)



On Sat, 16 May 2020 at 08:02, Matt Newville <newville at cars.uchicago.edu>
wrote:

> Hi Andrea,
>
> On Fri, May 15, 2020 at 10:42 AM Andrea Gavana <andrea.gavana at gmail.com>
> wrote:
>
>> On Fri, 15 May 2020 at 17.21, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, May 14, 2020 at 3:55 AM Andrew Nelson <andyfaff at gmail.com>
>>> wrote:
>>>
>>>> As a global optimizer in the first instance it would probably be added
>>>> as a separate function (`particle_swarm`), rather than be added as a method
>>>> to minimize. I'm not familiar with particle swarm, are there various
>>>> types/flavours of the approach?
>>>>
>>>> The general route to adding new global optimizers is:
>>>>
>>>
>>> Thanks for this detailed reply Andrew. I think it's worth adding this to
>>> the new section of the developer guide once
>>> https://github.com/scipy/scipy/pull/12069 is merged.
>>>
>>> One thing to ask additionally here is: are particle swarm methods in
>>> scope? It would be good to compare this proposed method to for example
>>> what's currently available in PyGMO (https://esa.github.io/pygmo/).
>>>
>>> Cheers,
>>> Ralf
>>>
>>
>> My apologies for pitching in, I’m usually just an observer. I’m sure some
>> of you remember the global optimization benchmarks I once posted - and I
>> believe they have been updated, adapted and integrated in scipy:
>>
>> http://infinity77.net/global_optimization/multidimensional.html
>>
>> In the set of optimizers, a particle swarm algorithm was actually there,
>> and specifically the one from:
>>
>> http://www.norg.uminho.pt/aivaz/pswarm/
>>
>> And it turned out to be not so bad in the end. My impression is that
>> there might be scope for a particle swarm algorithm in scipy: I’ve used a
>> few variants of them in the past where many other optimization procedures
>> failed. But then I’m just a lurker so please take my opinion with a grain
>> of salt :-).
>>
>> Andrea.
>>
>
> Thanks - I think your benchmarks are very informative and useful.  FWIW,
> we have implemented the `ampgo` solver to lmfit (
> https://lmfit.github.io/lmfit-py/fitting.html#lmfit.minimizer.Minimizer.ampgo),
> mainly based on your results.
>
> It looks to me like the next most useful to add would be `sce`.   That is,
> it appears to be roughly "second best" and pretty good at being disjoint to
> `ampgo`, `basinhopping`, and `diffev`.   Do you have code for this `sce`
> method that you could share?  I'm afraid I know absolutely nothing about
> this method.   I must admit that it was only from your work that I learned
> about `ampgo` too.
>
> Based on your results, it seems to me that adding `ampgo` and `sce` to
> scipy.optimize would be higher priorities than adding the particle smarm
> optimizer.   Sure, every method has value, but using your benchmarks might
> be a good way to prioritize which methods to add.
>
> Cheers, and thanks again,
>
> --Matt
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>


-- 
_____________________________________
Dr. Andrew Nelson


_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200516/26dc6474/attachment-0001.html>