[SciPy-User] (Possible) new optimization routines - scipy.optimize

[josef.pktd at gmail.com]

On Mon, Feb 18, 2013 at 11:41 AM, Andrea Gavana <andrea.gavana at gmail.com> wrote:
> On 14 February 2013 22:53,  <josef.pktd at gmail.com> wrote:
>> On Thu, Feb 14, 2013 at 4:01 PM, Andrea Gavana <andrea.gavana at gmail.com> wrote:
>>> Hi All,
>>>
>>>     as my team and I are constantly facing very hard/complex numerical
>>> optimization problems, I have taken a look at the various *global*
>>> optimization routines available in Python and I thought I could throw
>>> in a couple of algorithms I implemented, mostly drawing from my
>>> previous thesis work.
>>>
>>> I have implemented two routines (based on numpy and scipy), namely:
>>>
>>> - AMPGO: Adaptive Memory Programming for Global Optimization: this is my Python
>>>   implementation of the algorithm described here:
>>>
>>>   http://leeds-faculty.colorado.edu/glover/fred%20pubs/416%20-%20AMP%20(TS)%20for%20Constrained%20Global%20Opt%20w%20Lasdon%20et%20al%20.pdf
>>>
>>>   I have added a few improvements here and there based on my Master Thesis work
>>>   on the standard Tunnelling Algorithm of Levy, Montalvo and Gomez.
>>
>> This could also be a good addition. similar to basinhopping.
>> >From my perspective, this kind of global optimizers are the most
>> promising, (compared to the evolutionary, ...)
>>
>> >From a quick browse: Is the local optimizer fixed to a specific one,
>> or can it be any available solver as in basinhopping?
>>
>> The only thing I might worry about that it only has 6 citations in
>> Google Scholar (which might not mean much if the optimizer is not
>> widely available).
>> Given that there seem to be many variations of this kind of
>> optimizers, it might be good to have some background on comparison
>> with similar optimizers.
>>
>> If you have other comparisons of similar optimizers, it would be
>> useful to see them. Also given that you have a large benchmark suite,
>> you could compare it with the new basinhopping in scipy.optimize.
>
> OK, I have run the test suite with basinhopping as well, even though I
> had to modify it a bit to support the maximum number of functions
> evaluations and the tolerance on achieving the global optimum.

Anything that might be useful to incorporate in the scipy version?

>
> Results are here:
>
> - N-D : http://infinity77.net/global_optimization/multidimensional.html
> - 1-D : http://infinity77.net/global_optimization/univariate.html
>
> Overall it appears to have average performances, even though I must
> stress again that, while my test suite is relatively large, it only
> encompasses low-dimensional problem (1-10 variables) and my
> stopping/convergence criteria may not be applicable to everyone else's
> needs.

Interesting results, AMPGO looks good.

I'm a bit surprised that the number of function evaluations of
basinhopping is relatively large.
(One possible difference to your benchmark is that with smooth models
and numerical derivatives, a more efficient local optimizer could be
chosen.)

two things I saw when I looked at your benchmark before:

DE has several cases that are complementary to AMPGO, it might be
interesting in applications to cross-check results across optimizers.

Why does DE and some other have success rate either 0 or 100 and
nothing in between?
I don't see a reason what could cause this result.

Thanks

Josef


>
>
> Andrea.
>
> "Imagination Is The Only Weapon In The War Against Reality."
> http://www.infinity77.net
>
> # ------------------------------------------------------------- #
> def ask_mailing_list_support(email):
>
>     if mention_platform_and_version() and include_sample_app():
>         send_message(email)
>     else:
>         install_malware()
>         erase_hard_drives()
> # ------------------------------------------------------------- #
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user