[SciPy-dev] Making numpy test work in scikits

[David Cournapeau]

Matthieu Brucher wrote:
> Hi,
>
> I have to say that the fact that not all test_*.py are parsed for 
> tests annoys me in Numpy. If you want to test a file something.py, you 
> have to create a tests/test_something.py, you can't use an other name. 
> And when you want to test more than just a part of your code - like 
> for the optimizers, I want to test several modules together -, those 
> tests are not part of the test suite that Numpy creates.
You can change the convention for the names of the NumpyTest modules (at 
least, it looks like, having briefly read the code), but I guess 
everybody does as I did: copy the code from another toolbox, without 
thinking too much.

I don't understand what you mean by tests which are not part of the 
numpy test suite, though. For me, the problem is that there is no way 
(or more precisely, I didn't a find a way) to always get a testsuite 
from NumpyTest. Typically, my method works if you test everything, but 
does not work if you want to test some part of it.

For me, the point of numpy tests is to have common conventions:
    - every package define a test function which runs all the tests 
(from scipy.foo import test)
    - all the tests are in well defined places in the file layout.

Maybe there is room for improvement here: simplifying NumpyTestCase (why 
definining check_ function, for example ?), and making it a bit more 
flexible so that we can request a subset of the testsuite for a big 
package (for the package foo, getting a TestSuite containing all the 
tests in foo.bar.toto).
> Thus, I created a file that executes all test_*.py, but it cannot work 
> with setuptools.
>
> But I thought that this worked (I think I've used it before) :
>
> def test():
>   from numpy.testing import NumpyTest
>   return NumpyTest().test()
>
> in the __init__.py of a module that must be tested, doesn't it ?
I don't think it works because the function passed to setuptools needs 
to return a testsuite (that's the core problem: NumpyTest sometimes run 
tests, sometimes returns the testsuite).

David