[SciPy-Dev] CUTEst in Scipy

[Antonio Ribeiro]

Thank you for the insightful answer Nikolay! I agree with
you that include CUTEst dependency to Scipy could
be troublesome, and furthermore it could be very painful
to make this interface work well for more than one
operational systems... The solution of implementing
and store the benchmark somewhere else is not
ideal but seems a good option because it make things
simpler.

About the use of NIST test suite: Thank you for the suggestion
Andrew, but I don't think it is a good test suit for the methods
I intend to implement.


2017-03-20 19:23 GMT-03:00 <scipy-dev-request at scipy.org>:

> Send SciPy-Dev mailing list submissions to
>         scipy-dev at scipy.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.scipy.org/mailman/listinfo/scipy-dev
> or, via email, send a message with subject or body 'help' to
>         scipy-dev-request at scipy.org
>
> You can reach the person managing the list at
>         scipy-dev-owner at scipy.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-Dev digest..."
>
>
> Today's Topics:
>
>    1. CUTEst in Scipy (Antonio Ribeiro)
>    2. Re: CUTEst in Scipy (Nikolay Mayorov)
>    3. Re: CUTEst in Scipy (Andrew Nelson)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 20 Mar 2017 15:24:14 -0300
> From: Antonio Ribeiro <antonior92 at gmail.com>
> To: scipy-dev at scipy.org
> Subject: [SciPy-Dev] CUTEst in Scipy
> Message-ID:
>         <CAFStupFHFEk-JpGFgPikTvi5D+zq9VUG_K94k=7Pu+A5X79zPQ at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hello,
>
> I am developing my google of summer proposal about constrained optimisation
> in Scipy
> and it will be very important to have a good set of benchmarks.
>
> There is a great library with diverse optimisation benchmarks called CUTEst
> <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is under LGPL
> 2.1 license.
>
> This CUTEst  library include a huge amount of problem sets
> and is often used in optimisation papers. Furthermore, many of the
> available
> optimisation software provide some interface to it. I am interested in
> using problems from this set in my project and I want to ask how
>  should I proceed?
>
> 1) Select a subset of tests from the CUTEst library and implement them in
> native python under scipy.
>
> 2) Include some interface to CUTEst in scipy. By what I understand LGPL
> license is
> more restrictive than BSD-3 used by Scipy. In this case, could we ask for
> permission?
>
> 3) There is an interface for CUTEst implemented in the library pyOpus
> (under GPL3 license)
> <http://fides.fe.uni-lj.si/pyopus/download1.html>.  Since this is a
> library
> external to Scipy (with an incompatible license) how should I proceed in
> this case: can I make my benchmarks dependent on it? Can we ask permission
> for include this interface in Scipy?
>
> Any suggestions on how to proceed and what to include in my proposal is
> very welcome.
>
> Ant?nio
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://mail.scipy.org/pipermail/scipy-dev/
> attachments/20170320/f2417724/attachment-0001.html>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 21 Mar 2017 00:26:55 +0500
> From: Nikolay Mayorov <nikolay.mayorov at zoho.com>
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Subject: Re: [SciPy-Dev] CUTEst in Scipy
> Message-ID: <15aed2fc740.ec8f556f6477.4506434177608850793 at zoho.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi!
>
>
>
> I heard about CUTEst, but it looked quite complex so I decided not to deal
> with it. To your options:
>
>
>
> 1) It might be the most practical option, considering you'll have other
> (main) parts of work to do. Also it's not clear whether to include test
> problems in scipy or store them somewhere else. My opinion is that having a
> collection of problems right in scipy.optimize is good. I don't exactly
> remember why we decided not to include test problems for least_squares, I
> guess because it required many considerations. In theory there are ASV
> benchmarks in scipy, but they don't feel adequate for benchmarking
> optimization problems. So developing benchmarking infrastructure in
> scipy.optimize might be a good side project, my concern is that it will
> distract you from the main path greatly.
>
>
>
> To sum up: the simplest way is to pick some problems, write your own
> benchmark suite and store it somewhere else. This way you will be able to
> focus on the algorithms. Other options are possible if you have a really
> good idea how to do them and ready to devote time to it.
>
>
>
> 2) My understanding is that you don't need any permissions as you won't
> use CUTEst code (someone could correct me). As I see it: your add an
> interface into scipy, a user install CUTEst and use this interface to work
> with CUTEst. It doesn't seem very practical, because installing CUTEst
> looks like a pain. Do you agree or I am mistaken?
>
>
>
> 3) I guess you can ask an author to reuse his code in scipy to implement
> an interface to CUTEst.
>
>
>
> Generally about 2 and 3: it might be a good idea to use CUTEst for your
> internal benchmarking during the development and optionally you can add an
> interface to CUTEst into scipy. But I think it will be hardly ever used
> after that.
>
>
>
> To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking
> seems unpractical. It would be useful if you will actually try to work with
> CUTEst, maybe it will turn out to be not that difficult. In such case many
> of my points are not valid.
>
>
>
> I hope you'll be able to make some sense from my rambling.
>
>
> Nikolay.
>
>
>
>
> ---- On Mon, 20 Mar 2017 23:24:14 +0500 Antonio Ribeiro &
> lt;antonior92 at gmail.com> wrote ----
>
>
>
>
> Hello,
>
>
>
> I am developing my google of summer proposal about constrained
> optimisation in Scipy
>
> and it will be very important to have a good set of benchmarks.
>
>
>
> There is a great library with diverse optimisation benchmarks called
> CUTEst <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It
> is under LGPL 2.1 license.
>
>
>
> This CUTEst  library include a huge amount of problem sets
>
> and is often used in optimisation papers. Furthermore, many of the
> available
>
> optimisation software provide some interface to it. I am interested in
>
> using problems from this set in my project and I want to ask how
>
>  should I proceed?
>
>
>
>
> 1) Select a subset of tests from the CUTEst library and implement them in
> native python under scipy.
>
>
>
> 2) Include some interface to CUTEst in scipy. By what I understand LGPL
> license is
>
> more restrictive than BSD-3 used by Scipy. In this case, could we ask for
> permission?
>
>
>
> 3) There is an interface for CUTEst implemented in the library pyOpus
> (under GPL3 license)
>
> <http://fides.fe.uni-lj.si/pyopus/download1.html>.  Since this is a
> library external to Scipy (with an incompatible license) how should I
> proceed in this case: can I make my benchmarks dependent on it? Can we ask
> permission for include this interface in Scipy?
>
>
>
>
>
> Any suggestions on how to proceed and what to include in my proposal is
> very welcome.
>
>
>
> Ant?nio
>
>
>
> _______________________________________________
>
> SciPy-Dev mailing list
>
> SciPy-Dev at scipy.org
>
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://mail.scipy.org/pipermail/scipy-dev/
> attachments/20170321/751cb63f/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 21 Mar 2017 09:21:59 +1100
> From: Andrew Nelson <andyfaff at gmail.com>
> To: SciPy Developers List <scipy-dev at scipy.org>
> Subject: Re: [SciPy-Dev] CUTEst in Scipy
> Message-ID:
>         <CAAbtOZc2JnOvgvzH4jEB3mpLBm_YZNCj9xy1q_PG2YN2+EMD4w at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> There are some least squares problems from the NIST test suite in the
> benchmarks/benchmarks/go_benchmark_functions, but they're couched in terms
> of a simple scalar (chi2) minimisation problem, and they are all bounded
> problems.
>
>
> On 21 March 2017 at 06:26, Nikolay Mayorov <nikolay.mayorov at zoho.com>
> wrote:
>
> > Hi!
> >
> > I heard about CUTEst, but it looked quite complex so I decided not to
> deal
> > with it. To your options:
> >
> > 1) It might be the most practical option, considering you'll have other
> > (main) parts of work to do. Also it's not clear whether to include test
> > problems in scipy or store them somewhere else. My opinion is that
> having a
> > collection of problems right in scipy.optimize is good. I don't exactly
> > remember why we decided not to include test problems for least_squares, I
> > guess because it required many considerations. In theory there are ASV
> > benchmarks in scipy, but they don't feel adequate for benchmarking
> > optimization problems. So developing benchmarking infrastructure in
> > scipy.optimize might be a good side project, my concern is that it will
> > distract you from the main path greatly.
> >
> > To sum up: the simplest way is to pick some problems, write your own
> > benchmark suite and store it somewhere else. This way you will be able to
> > focus on the algorithms. Other options are possible if you have a really
> > good idea how to do them and ready to devote time to it.
> >
> > 2) My understanding is that you don't need any permissions as you won't
> > use CUTEst code (someone could correct me). As I see it: your add an
> > interface into scipy, a user install CUTEst and use this interface to
> work
> > with CUTEst. It doesn't seem very practical, because installing CUTEst
> > looks like a pain. Do you agree or I am mistaken?
> >
> > 3) I guess you can ask an author to reuse his code in scipy to implement
> > an interface to CUTEst.
> >
> > Generally about 2 and 3: it might be a good idea to use CUTEst for your
> > internal benchmarking during the development and optionally you can add
> an
> > interface to CUTEst into scipy. But I think it will be hardly ever used
> > after that.
> >
> > To sum the idea: adding CUTEst dependency for scipy.optimize benchmarking
> > seems unpractical. It would be useful if you will actually try to work
> with
> > CUTEst, maybe it will turn out to be not that difficult. In such case
> many
> > of my points are not valid.
> >
> > I hope you'll be able to make some sense from my rambling.
> >
> > Nikolay.
> >
> >
> > ---- On Mon, 20 Mar 2017 23:24:14 +0500 *Antonio Ribeiro
> > <antonior92 at gmail.com <antonior92 at gmail.com>>* wrote ----
> >
> > Hello,
> >
> > I am developing my google of summer proposal about constrained
> > optimisation in Scipy
> > and it will be very important to have a good set of benchmarks.
> >
> > There is a great library with diverse optimisation benchmarks called
> > CUTEst <https://ccpforge.cse.rl.ac.uk/gf/project/cutest/wiki/>. It is
> > under LGPL 2.1 license.
> >
> > This CUTEst  library include a huge amount of problem sets
> > and is often used in optimisation papers. Furthermore, many of the
> > available
> > optimisation software provide some interface to it. I am interested in
> > using problems from this set in my project and I want to ask how
> >  should I proceed?
> >
> > 1) Select a subset of tests from the CUTEst library and implement them in
> > native python under scipy.
> >
> > 2) Include some interface to CUTEst in scipy. By what I understand LGPL
> > license is
> > more restrictive than BSD-3 used by Scipy. In this case, could we ask for
> > permission?
> >
> > 3) There is an interface for CUTEst implemented in the library pyOpus
> > (under GPL3 license)
> > <http://fides.fe.uni-lj.si/pyopus/download1.html>.  Since this is a
> > library external to Scipy (with an incompatible license) how should I
> > proceed in this case: can I make my benchmarks dependent on it? Can we
> ask
> > permission for include this interface in Scipy?
> >
> > Any suggestions on how to proceed and what to include in my proposal is
> > very welcome.
> >
> > Ant?nio
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >
>
>
> --
> _____________________________________
> Dr. Andrew Nelson
>
>
> _____________________________________
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://mail.scipy.org/pipermail/scipy-dev/
> attachments/20170321/de9dde91/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
> ------------------------------
>
> End of SciPy-Dev Digest, Vol 161, Issue 28
> ******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20170321/6e5bde70/attachment.html>