[SciPy-Dev] CUTEst in Scipy

[Ralf Gommers]

(when replying to a thread, manually change scipy-dev at scipy.org to
scipy-dev at python.org)



On Tue, Mar 21, 2017 at 8:26 AM, Nikolay Mayorov <nikolay.mayorov at zoho.com>
wrote:

> Hi!
>
> I heard about CUTEst, but it looked quite complex so I decided not to deal
> with it. To your options:
>
> 1) It might be the most practical option, considering you'll have other
> (main) parts of work to do. Also it's not clear whether to include test
> problems in scipy or store them somewhere else. My opinion is that having a
> collection of problems right in scipy.optimize is good. I don't exactly
> remember why we decided not to include test problems for least_squares, I
> guess because it required many considerations. In theory there are ASV
> benchmarks in scipy, but they don't feel adequate for benchmarking
> optimization problems.
>

Why do you say that? The optimize benchmarks are quite extensive, and came
from http://infinity77.net/global_optimization/index.html which is doing
exactly what you say our asv benchmarks are not adequate for. If it's about
usability/runtime, how about just fixing that? That plus adding some
problems relevant for Antonio's GSoC project seems to make the most sense
to me.

Ralf



> So developing benchmarking infrastructure in scipy.optimize might be a
> good side project, my concern is that it will distract you from the main
> path greatly.
>
> To sum up: the simplest way is to pick some problems, write your own
> benchmark suite and store it somewhere else. This way you will be able to
> focus on the algorithms. Other options are possible if you have a really
> good idea how to do them and ready to devote time to it.
>
> 2) My understanding is that you don't need any permissions as you won't
> use CUTEst code (someone could correct me). As I see it: your add an
> interface into scipy, a user install CUTEst and use this interface to work
> with CUTEst. It doesn't seem very practical, because installing CUTEst
> looks like a pain. Do you agree or I am mistaken?
>
> 3) I guess you can ask an author to reuse his code in scipy to implement
> an interface to CUTEst.
>
> Generally about 2 and 3: it might be a good idea to use CUTEst for your
> internal benchmarking during the development and optionally you can add an
> interface to CUTEst into scipy. But I think it will be hardly ever used
> after that.
>
> To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking
> seems unpractical. It would be useful if you will actually try to work with
> CUTEst, maybe it will turn out to be not that difficult. In such case many
> of my points are not valid.
>
> I hope you'll be able to make some sense from my rambling.
>
> Nikolay.
>
>
> ---- On Mon, 20 Mar 2017 23:24:14 +0500 *Antonio Ribeiro
> <antonior92 at gmail.com <antonior92 at gmail.com>>* wrote ----
>
> Hello,
>
> I am developing my google of summer proposal about constrained
> optimisation in Scipy
> and it will be very important to have a good set of benchmarks.
>
> There is a great library with diverse optimisation benchmarks called
> CUTEst <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is
> under LGPL 2.1 license.
>
> This CUTEst  library include a huge amount of problem sets
> and is often used in optimisation papers. Furthermore, many of the
> available
> optimisation software provide some interface to it. I am interested in
> using problems from this set in my project and I want to ask how
>  should I proceed?
>
> 1) Select a subset of tests from the CUTEst library and implement them in
> native python under scipy.
>
> 2) Include some interface to CUTEst in scipy. By what I understand LGPL
> license is
> more restrictive than BSD-3 used by Scipy. In this case, could we ask for
> permission?
>
> 3) There is an interface for CUTEst implemented in the library pyOpus
> (under GPL3 license)
> <http://fides.fe.uni-lj.si/pyopus/download1.html>.  Since this is a
> library external to Scipy (with an incompatible license) how should I
> proceed in this case: can I make my benchmarks dependent on it? Can we ask
> permission for include this interface in Scipy?
>
> Any suggestions on how to proceed and what to include in my proposal is
> very welcome.
>
> Antônio
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20170326/06e88fa6/attachment.html>