[SciPy-Dev] Consideration of differential evolution minimizer being added to scipy.optimize.

[Ralf Gommers]

On Sat, Mar 8, 2014 at 10:55 PM, Andrew Nelson <andyfaff at gmail.com> wrote:

> Dear all,
> I've put some more work into the differential evolution (another
> global optimizer) code I would like to contribute to scipy.optimize.
> Since this is the first time I'm contributing to scipy I'd would like
> a code review, with a view to putting in a pull request in the near
> future.
>

> The changes/additions I've made are at (under files changed):
> https://github.com/andyfaff/scipy/compare/scipy:master...master
>

Could you send a PR? That's easier to comment on, and will be preserved
better for future reference.

Question that the docstring doesn't answer:
- why are there so many options for `DEstrategy` (and what's the default)?
- `mutation` and `regeneration` aren't explained, do I need to tweak those
as a user?

Some minor comments:
- please reference the original paper (format as in
https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt)
instead of linking to someone's homepage.
- there's ``**options`` in the signature, but it's not explained and
doesn't appear to be needed.
- tests: don't use plain assert, use np.testing.assert_ instead
- tests: don't import unittest, only numpy.testing
- tests: test cases shouldn't have docstrings because that messes up nose
output, replace with comments



> Some of the comments when I first discussed this were for benchmarking
> DE to see if it had comparable performance to basinhopping.  To that
> end I've added some code and extra test functions
> (http://en.wikipedia.org/wiki/Test_functions_for_optimization)  for
> benchmarking global optimizers.  You can see the output of the
> benchmarking at: https://gist.github.com/andyfaff/9439229
>

So it looks like on average DE has worse performance, but there are
problems where it does significantly better than basinhopping. That's also
the conclusion one could draw from
http://infinity77.net/global_optimization/multidimensional.html. Which may
be an OK motivation to add it.

Cheers,
Ralf



> >From the test functions differential_evolution doesn't do too badly,
> especially when no gradient is available for the function to be
> minimized.  Having said that, I do recognise that you sometimes need
> to tune the optimizer for a specific problem.
>
> My specific motivation for adding differential_evolution is for
> curve_fitting.  For most of my problems function expense isn't
> enormous, so I don't regard number of function evaluations to be all
> that important.  In those cases finding the global minimum is the key.
>  I've tried comparing basinhopping to DE for harder curvefitting
> problems and in the example I tried the number of function evaluations
> was a lot smaller for DE
> (http://www.itl.nist.gov/div898/strd/index.html - Thurber).
> (One can always use Levenberg Marquardt to 'polish' the end result)
>
> Of course, there are problems where function expense is enormous.
> However, if that is truly horrendous one can always use the callback
> function to terminate if you think one is getting towards a solution.
>
> cheers,
> Andrew.
>
>
>
> --
> _____________________________________
> Dr. Andrew Nelson
>
>
> _____________________________________
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20140309/97caa4d8/attachment.html>