[Numpy-discussion] Test framework changes

[Alan McIntyre]

These changes are now checked in, with the changes Robert suggested:
numpy.test() returns a TextTestResult object again, and coverage (if
enabled) is limited to the package from which test is called.

The tests still fail because the buildbots don't have nose installed, though.

On Fri, Jun 20, 2008 at 5:01 PM, Alan McIntyre <alan.mcintyre at gmail.com> wrote:
> Hi all,
>
> Just wanted to get feedback about the following changes before I make
> them.  Please speak up if any of this seems objectionable to you.
>
> - The old test framework classes will be restored, but will not be
> used anywhere in NumPy's tests.  If your old tests still don't work
> with the restored classes, please let me know and I'll fix it.
>
> - Add a function to numpy.testing to execute a module's tests via the
> "if __name__ == '__main__'" hook.  It takes one optional argument, the
> name of the file to run (if not present, it uses the same method as
> NoseTester to look it up from the stack).  The intent is to make the
> boilerplate as simple as possible.
>
>  if __name__ == '__main__':
>      run_module_suite()
>
>  If somebody has a suggestion for a better name, I'd love to hear it.
> I didn't want "test" in the name, because then I have to explicitly
> tell nose to ignore it when it's looking for test functions.
>
> - Remove numpy/testing/pkgtester.py since it now just has one line of
> code that imports NoseTester (as Tester) from nosetester.py (it used
> to create a null tester if nose wasn't present, but this was removed
> by porting of r4424 from SciPy).  NoseTester will still be made
> available as numpy.testing.Tester.
>
> - numpy.test (and all the other test functions in subpackages) will
> take the following positional arguments: label, verbose, extra_argv,
> doctests, coverage (the same arguments as SciPy.test except for the
> new coverage option).  The old arguments can be passed in as keyword
> arguments (which seems to be how they were passed in in all the
> examples I could find), and they will be emulated as much as possible
> by the new test suite, but a deprecation warning will be issued.
>
> - numpy.test now returns an object with a wasSuccessful method; under
> the old test framework it returned a unittest._TextTestResult, but
> since nose.run only returns success/failure, I'm just wrapping this
> result in a dummy object to match the old behavior until I can figure
> out how to get a real _TextTestResult from nose.
>
> (The two changes to numpy.test should allow the buildbots to run the
> tests properly again)
>
> Thanks,
> Alan
>