[Numpy-discussion] Time for beta1 of NumPy 1.0
Sasha
ndarray at mac.com
Fri Jun 30 18:21:21 EDT 2006
"Software developers also use coverage testing in concert with
testsuites, to make sure software is actually good enough for a
release. " -- Gcov Manual
I think if we can improve the test coverage, it will speak volumes
about the quality of numpy.
Does anyone know if it is possible to instrument numpy libraries
without having to instrument python itself?
It would be nice to make the coverage reports easily available either
by including a generating script with the source distribution or by
publishing the reports for the releases.
On 6/30/06, Sasha <ndarray at mac.com> wrote:
> It is not as bad as I thought, but there is certainly room for improvement.
>
> File `numpy/core/src/multiarraymodule.c'
> Lines executed:63.56% of 3290
>
> File `numpy/core/src/arrayobject.c'
> Lines executed:59.70% of 5280
>
> File `numpy/core/src/scalartypes.inc.src'
> Lines executed:31.67% of 963
>
> File `numpy/core/src/arraytypes.inc.src'
> Lines executed:47.35% of 868
>
> File `numpy/core/src/arraymethods.c'
> Lines executed:57.65% of 739
>
>
>
> On 6/30/06, Sasha <ndarray at mac.com> wrote:
> > As soon as I sent out my 10% estimate, I realized that someone will
> > challenge it with a python level coverage statistics. My main concern
> > is not what fraction of numpy functions is called by unit tests, but
> > what fraction of special cases in the C code is exercised. I am not
> > sure that David's statistics even answers the first question - I would
> > guess it only counts statements in the pure python methods and
> > ignores methods implemented in C.
> >
> > Can someone post C-level statistics from gcov
> > <http://gcc.gnu.org/onlinedocs/gcc/Gcov.html> or a similar tool?
> >
> > On 6/30/06, David M. Cooke <cookedm at physics.mcmaster.ca> wrote:
> > > On Fri, 30 Jun 2006 12:35:35 -0400
> > > Sasha <ndarray at mac.com> wrote:
> > >
> > > > On 6/30/06, Fernando Perez <fperez.net at gmail.com> wrote:
> > > > > ...
> > > > > Besides, decent unit tests will catch these problems. We all know
> > > > > that every scientific code in existence is unit tested to the smallest
> > > > > routine, so this shouldn't be a problem for anyone.
> > > >
> > > > Is this a joke? Did anyone ever measured the coverage of numpy
> > > > unittests? I would be surprized if it was more than 10%.
> > >
> > > A very quick application of the coverage module, available at
> > > http://www.garethrees.org/2001/12/04/python-coverage/
> > > gives me 41%:
> > >
> > > Name Stmts Exec Cover
> > > ---------------------------------------------------
> > > numpy 25 20 80%
> > > numpy._import_tools 235 175 74%
> > > numpy.add_newdocs 2 2 100%
> > > numpy.core 28 26 92%
> > > numpy.core.__svn_version__ 1 1 100%
> > > numpy.core._internal 99 48 48%
> > > numpy.core.arrayprint 251 92 36%
> > > numpy.core.defchararray 221 58 26%
> > > numpy.core.defmatrix 259 186 71%
> > > numpy.core.fromnumeric 319 153 47%
> > > numpy.core.info 3 3 100%
> > > numpy.core.ma 1612 1145 71%
> > > numpy.core.memmap 64 14 21%
> > > numpy.core.numeric 323 138 42%
> > > numpy.core.numerictypes 236 204 86%
> > > numpy.core.records 272 32 11%
> > > numpy.dft 6 4 66%
> > > numpy.dft.fftpack 128 31 24%
> > > numpy.dft.helper 35 32 91%
> > > numpy.dft.info 3 3 100%
> > > numpy.distutils 13 9 69%
> > > numpy.distutils.__version__ 4 4 100%
> > > numpy.distutils.ccompiler 296 49 16%
> > > numpy.distutils.exec_command 409 27 6%
> > > numpy.distutils.info 2 2 100%
> > > numpy.distutils.log 37 18 48%
> > > numpy.distutils.misc_util 945 174 18%
> > > numpy.distutils.unixccompiler 34 11 32%
> > > numpy.dual 41 27 65%
> > > numpy.f2py.info 2 2 100%
> > > numpy.lib 30 28 93%
> > > numpy.lib.arraysetops 121 59 48%
> > > numpy.lib.function_base 501 70 13%
> > > numpy.lib.getlimits 76 61 80%
> > > numpy.lib.index_tricks 223 56 25%
> > > numpy.lib.info 4 4 100%
> > > numpy.lib.machar 174 154 88%
> > > numpy.lib.polynomial 357 52 14%
> > > numpy.lib.scimath 51 19 37%
> > > numpy.lib.shape_base 220 24 10%
> > > numpy.lib.twodim_base 77 51 66%
> > > numpy.lib.type_check 110 75 68%
> > > numpy.lib.ufunclike 37 24 64%
> > > numpy.lib.utils 42 23 54%
> > > numpy.linalg 5 3 60%
> > > numpy.linalg.info 2 2 100%
> > > numpy.linalg.linalg 440 71 16%
> > > numpy.random 10 6 60%
> > > numpy.random.info 4 4 100%
> > > numpy.testing 3 3 100%
> > > numpy.testing.info 2 2 100%
> > > numpy.testing.numpytest 430 214 49%
> > > numpy.testing.utils 151 62 41%
> > > numpy.version 7 7 100%
> > > ---------------------------------------------------
> > > TOTAL 8982 3764 41%
> > >
> > > (I filtered out all the *.tests.* modules). Note that you have to import
> > > numpy after starting the coverage, because we use a lot of module-level code
> > > that wouldn't be caught otherwise.
> > >
> > > --
> > > |>|\/|<
> > > /--------------------------------------------------------------------------\
> > > |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/
> > > |cookedm at physics.mcmaster.ca
> > >
> >
>
More information about the NumPy-Discussion
mailing list