[SciPy-User] Minimizing Monte Carlo simulation function

[Matt Newville]

On Wed, Sep 24, 2014 at 10:19 AM, Sturla Molden <sturla.molden at gmail.com>
wrote:

> Matt Newville <newville at cars.uchicago.edu> wrote:
>
> > I think no one would disagree that scipy.optimize needs more (and better)
> > optimizers, but deprecating those that work poorly (anneal) when others
> > that work better (basinhopping) are available seems sensible to me.
>
> Which optimizer works better depends on the problem.
>

Andrea tested 202 example problems from the literature, with 100 random
starting values for each.   For the overwhelming majority (around 85%) of
the problems, simulated annealing never found the correct solution.  That
alone might justify being deprecated (at least in the sense of
disapproval).   Perhaps there are flaws in anneal() that could be
improved.   I don't know.   But it seems that it is not going to work for
many cases.

OTOH, For about 5% of the problems, anneal found the correct solution from
more than 50% of the starting values, and for about 5% of the problems,
anneal found the correct solution more frequently than basin hopping.  For
3% of the problems, simulated annealing out-performed both basin hopping
and AMPGO.   With basin hopping included, I think it is perfectly
reasonable to recommend basin hopping over simulated annealing.  Ideally,
AMPGO and other routines could be added.

Simulated Annealing is a well known algorithm and it makes sence to keep
> it, at least for reference.
>

I think it's reasonable to expect a "fitness for purpose".  Knowing that it
gets the correct solution in fewer than 10% of the test problems can't
inspire great confidence to anyone.  If anneal is un-deprecated
(re-approved?), I would suggest that its miserable track record be
documented in the top level of scipy.optimize, where the unsuspecting user
might otherwise see it listed as one of the few Global Optimizers in scipy,
and be led to the mistaken belief that its results are reliable.

--Matt Newville
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140924/738f54ad/attachment.html>