[SciPy-Dev] SciPy-Dev Digest, Vol 161, Issue 28

[Matt Newville]

Date: Tue, 21 Mar 2017 09:21:59 +1100
> From: Andrew Nelson <andyfaff at gmail.com>
> To: SciPy Developers List <scipy-dev at scipy.org>
> Subject: Re: [SciPy-Dev] CUTEst in Scipy
> Message-ID:
>         <CAAbtOZc2JnOvgvzH4jEB3mpLBm_YZNCj9xy1q_PG2YN2+EMD4w at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> There are some least squares problems from the NIST test suite in the
> benchmarks/benchmarks/go_benchmark_functions, but they're couched in terms
> of a simple scalar (chi2) minimisation problem, and they are all bounded
> problems.
>
>
The NIST StRD test suite (
http://www.itl.nist.gov/div898/strd/nls/nls_main.shtml) could be considered
for inclusion as a test suite for scipy.optimize.  These are generally
small but non-trivial problems for unconstrained non-linear optimization.
Each of the 20+ data sets comes with 2 sets of starting values for the
variables and certified values for both best fit values and estimated
uncertainties.  I believe that none of the optimizers in scipy.optimize
will get the right answer to published precision in every case, though
leastsq and least_squares can solve most of these problems reasonably well
from at least 1 of the 2 starting points.  This may imply that these
problems are not "sparse" in the sense of what (as far as I understand) the
GSOC project intends to focus on.  But, they are a good set of test cases
to include in any assessment of an optimization algorithm.

The NIST StRD datasets are public domain.  There is existing test code in
lmfit to use these data sets with most of the scipy optimizers.  This could
easily be modified to be part of scipy.   See
https://github.com/lmfit/lmfit-py/tree/master/NIST_STRD and
https://github.com/lmfit/lmfit-py/blob/master/tests/test_NIST_Strd.py for
more details.  Also, note that the tests focus only on the quality of the
results.  They may record number of iterations, but make no attempt to
benchmark the runtime.

I think that migrating this test suite to scipy would not be a huge
undertaking, though I cannot say whether that is an appropriate use of time
for the proposed GSOC project.

Cheers,

--Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20170320/a33d29a6/attachment.html>