[Numpy-discussion] known failure test decorator not treated as expected by nose

[Ralf Gommers]

On Fri, Oct 8, 2010 at 1:09 AM, Jonathan March <jmarch at enthought.com> wrote:

> On Wed, Oct 6, 2010 at 6:17 PM,  <josef.pktd at gmail.com> wrote:
> > On Wed, Oct 6, 2010 at 7:08 PM, Jonathan March <jmarch at enthought.com>
> wrote:
> >> It appears that the numpy testing decorators for skipping and for
> >> known failure should behave similarly to each other, at least from
> >> their descriptions here:
> >>
> http://projects.scipy.org/numpy/wiki/TestingGuidelines#known-failures-skipping-tests
> >>
> >> Yet in the following example, run under nose, they behave quite
> differently:
> >>
> >> ====
> >> from numpy.testing import  dec
> >>
> >> @dec.knownfailureif( True, "known failure test message")
> >> def test_will_fail():
> >>    pass
> >>
> >> @dec.skipif( True, "skip test message")
> >> def test_to_skip():
> >>    pass
> >> ====
> >>
> >> The skipped test is marked as "S" as expected, but when the
> >> KnownFailureTest exception is raised, nose handles it as an error
> >> (marked "E" rather than "K" as expected, and printing the stack
> >> trace.)
> >>
> >> It's also interesting that nose has a --no-skip option but no
> >> corresponding option for ignoring known failure decorators.
> >>
> >> Why the discrepancy? Who is out of step with whom? This is with numpy
> >> 1.4.0 and nosetests 0.11.3
> >
> > just as additional information
> >
> > This only happens on the command line, with nosetests packagename
> > but not with the function, packagename.test()
> >
>

Knownfailure is a numpy-specific plugin, which needs to be loaded for tests
decorated with @dec.knownfailureif to be marked as K instead of E. This
plugin loading is done by the NoseTester class (defined in
numpy/testing/nosetester.py) which is used when tests are run through
numpy.test().

Using 'nosetests' on the command line skips this plugin loading, hence the
error. Nose does have a "skip" plugin so that never gives an error.


> > Josef
>
>
> Numpy issue, nose issue, or user issue?
>

User issue.

Cheers,
Ralf


>
>
> >> Thanks,
> >> Jonathan March
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20101009/091246fa/attachment.html>