[SciPy-Dev] Extending scipy.optimize.minimize with particle swarm

[Ralf Gommers]

On Thu, May 14, 2020 at 3:55 AM Andrew Nelson <andyfaff at gmail.com> wrote:

> As a global optimizer in the first instance it would probably be added as
> a separate function (`particle_swarm`), rather than be added as a method to
> minimize. I'm not familiar with particle swarm, are there various
> types/flavours of the approach?
>
> The general route to adding new global optimizers is:
>

Thanks for this detailed reply Andrew. I think it's worth adding this to
the new section of the developer guide once
https://github.com/scipy/scipy/pull/12069 is merged.

One thing to ask additionally here is: are particle swarm methods in scope?
It would be good to compare this proposed method to for example what's
currently available in PyGMO (https://esa.github.io/pygmo/).

Cheers,
Ralf



> - is it a recognised solver approach that is proven in the scientific
> literature?
> - run it against the global benchmark suite, which would involve modifying
> https://github.com/scipy/scipy/blob/master/benchmarks/benchmarks/optimize.py (do
> this before you do any modification of scipy library code). Running those
> global benchmarks for the `particle_swarm` function would check that the
> solver is performant enough. By performant we mean that the percentage
> success rate against the problems is as good as the other global solvers.
> Also of interest is the average number of function evaluations to reach
> that success rate. (Time taken is probably related to number of function
> evaluations). Ideally the function fills holes that the other solvers can't
> handle.
>
> If it's performant then we can go further towards adding it to scipy
> (these steps would be after a further discussion on this list):
>
> - code needs to be compatible with the scipy licence (i.e. no GPL/LGPL)
> - the changes need a comprehensive test suite.
> - I strongly suggest that you have a public `particle_swarm` function, and
> a private ParticleSwarm class that does the solving behind the scenes. If
> possible make the ParticleSwarm object an iterator with a __next__ method
> so that individual iteration steps can be taken. See
> https://github.com/scipy/scipy/blob/master/scipy/optimize/_differentialevolution.py as
> an example. A full solve could be done with a `solve` method.
> - keep the API as close as possible to other optimizers, i.e. same
> parameter names, same keyword names, parameter order is similar to other
> minimizers.
> - differential_evolution benefitted from offering a reduced set of
> functionality when it was first added. Additional features came afterwards
> and the delayed introduction of them meant that the API/design of the
> solver benefitted from a lot of thought over a prolonged period.
> - if `particle_swarm` uses random number generation then the solver should
> be able to generate random numbers with either `np.random.RandomState` or
> `np.random.Generator` (
> https://github.com/scipy/scipy/blob/master/scipy/optimize/_differentialevolution.py#L560).
> This means using methods that belong to both type of objects.
> - the random number generation needs to be reproducible (function should
> have `seed` keyword).
> - total number of function evaluations needs to be tracked.
> - consider parallelisation aspects of your function. Can calls to the
> objective function be done in parallel? (`workers` keyword).
> - vectorisation is nice.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20200515/0e83e024/attachment.html>