[SciPy-Dev] Global Optimization Benchmarks

Daniel Schmitz danielschmitzsiegen at googlemail.com
Sat Jan 9 04:11:09 EST 2021


Thanks Andrea, will give biteopt a try. At first glance though, despite the
"sparse" documentation it should be possible to work after checking the
examples. Regarding a publication, I would not be so pessimistic: the sheer
amount of benchmarks you carried out is very interesting for the
optimization community!

The paper you attached is very interesting, Tomlab seems to be superior for
hard problems in general.

Best,
Daniel


On Sat, 9 Jan 2021 at 09:31, Andrea Gavana <andrea.gavana at gmail.com> wrote:

>
> On Sat, 9 Jan 2021 at 09.12, Andrea Gavana <andrea.gavana at gmail.com>
> wrote:
>
>> Hi Daniel,
>>
>> On Sat, 9 Jan 2021 at 08.37, Daniel Schmitz <
>> danielschmitzsiegen at googlemail.com> wrote:
>>
>>> Awesome work, Andrea!
>>>
>>
>> Thank you, I’m happy you liked it :-) . I have been told to try and
>> condense the work in a paper and submit it to the Journal of Global
>> Optimization, although I’m currently doubting that such a prestigious
>> publication will accept a publication from a nobody like me.
>>
>>
>>> Would it be possible for you to make your implementations of mcs and
>>> biteopt publicly available?
>>>
>>
>> For BiteOpt I might be able to - after a bit of polishing, nothing major.
>> For both I’ll have to ask for permission from my employer, as it was a
>> little tour the force getting MCS right.
>>
>
> That said, there is a publicly available Python wrapper for BiteOpt here:
>
> https://github.com/leonidk/biteopt
>
> I haven’t used it myself, I decided to rewrite the algorithm in Python as
> I wanted to understand what the code was doing and I wanted to add a couple
> of (minor) modifications in order to try and make the solver more efficient.
>
> Andrea.
>
>
>
>
>>
>> And more out of curiosity, not directly related to scipy: since you work
>>> in an industry setting, did you compare these open source optimizers
>>> against commercial ones like knitro?
>>>
>>
>> Unfortunately we don’t have any commercial solver available - as far as I
>> know - but I would love to redo the work using other solvers (open source
>> or not). The very interesting exercise from Sahinidis et al:
>>
>> https://link.springer.com/content/pdf/10.1007/s10898-012-9951-y.pdf
>>
>> Although limited to a single set of benchmarks, show that commercial
>> solvers like Tomlab are very efficient, tightly followed by MCS. It would
>> be nice to try and apply the solvers in KNITRO, Tomlab, GlobSol (others?)
>> to all the 16 benchmarks to see where open source stands, but I have no
>> idea how it could be done as I have no licenses for those.
>>
>> Andrea.
>>
>>
>>
>>> Cheers,
>>>
>>> Daniel
>>>
>>> On Fri, 8 Jan 2021 at 12:28, Ralf Gommers <ralf.gommers at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Jan 8, 2021 at 12:20 PM Andrea Gavana <andrea.gavana at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ralf,
>>>>>
>>>>> On Fri, 8 Jan 2021 at 12:15, Ralf Gommers <ralf.gommers at gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 8, 2021 at 11:35 AM Andrea Gavana <
>>>>>> andrea.gavana at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Ralf,
>>>>>>>
>>>>>>> On Fri, 8 Jan 2021 at 11:07, Ralf Gommers <ralf.gommers at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 8, 2021 at 10:21 AM Andrea Gavana <
>>>>>>>> andrea.gavana at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Dear SciPy Developers & Users,
>>>>>>>>>
>>>>>>>>>     long time no see :-) . I thought to start 2021 with a bit of a
>>>>>>>>> bang, to try and forget how bad 2020 has been... So I am happy to present
>>>>>>>>> you with a revamped version of the Global Optimization Benchmarks from my
>>>>>>>>> previous exercise in 2013.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Andrea, this is awesome! Thanks for sharing!
>>>>>>>>
>>>>>>>
>>>>>>> I am happy you like it :-) .
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> This could be really useful to link to and use as a guide for
>>>>>>>> providing recommendations for solvers to use in the scipy.optimize
>>>>>>>> tutorials. It's good to see that SciPy overall is much more competitive
>>>>>>>> than it was in 2013. Overall it seems SHGO is our most accurate solver, and
>>>>>>>> making it faster seems worthwhile. That shouldn't be very difficult, given
>>>>>>>> that it's all pure Python still.
>>>>>>>>
>>>>>>>
>>>>>>> I have to say that, compared to back in 2013, the addition of SHGO
>>>>>>> and DualAnnealing to SciPy has made the global optimization world in SciPy
>>>>>>> much more powerful, pretty much at the top of what can currently be done
>>>>>>> with open source solvers.
>>>>>>>
>>>>>>>
>>>>>>>> MCS isn't open source, but both DIRECT and BiteOpt are MIT-licensed
>>>>>>>> and seem the best candidates to be considered for inclusion in SciPy.
>>>>>>>>
>>>>>>>
>>>>>>> I couldn't find a license restriction for MCS, but maybe I haven't
>>>>>>> looked hard enough... Do you have a link for it? I am just curious.
>>>>>>>
>>>>>>
>>>>>> MCS itself doesn't contain any license information, but it depends on
>>>>>> MINQ which has a link in "All versions of MINQ are licensed" on this page:
>>>>>> https://www.mat.univie.ac.at/~neum/software/minq/. It's only free
>>>>>> for non-commercial use.
>>>>>>
>>>>>
>>>>>
>>>>> Ah, OK, thank you, I didn't think about that. Of course, assuming
>>>>> SciPy had another, different "bound constrained indefinite quadratic
>>>>> programming" module then we could easily swap it :-) .
>>>>>
>>>>
>>>> MCS and MINQ are from the same author, so I'd expect the same
>>>> restriction to apply to MCS though. We could ask for permission to license
>>>> all that under BSD/MIT, sometimes that works - the author seems like the
>>>> typical academic who doesn't understand open source licensing. In the past
>>>> we've had success with explaining; given how much extra exposure/users MCS
>>>> gets if it would be included in SciPy, it may be worth doing if someone is
>>>> motivated to work on integrating MCS into SciPy.
>>>>
>>>> Cheers,
>>>> Ralf
>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210109/601bd791/attachment-0001.html>


More information about the SciPy-Dev mailing list