[py-dev] running doctest-based tests

[Max Ischenko]

Holger,

> > My situation requires a custom representation of a failure instead of
plain
> > traceback.
> >
> > A solution might be to abstract the way failures are reported by moving
this
> > responsibility from TerminalSession to the Failure class, for example.
After
> > all, this even seems logical - different kind of failures may require
> > different representations and a Failure class is a natural place to hold
> > this information.  OTOH, the session classes *are* responsible for an
actual
> > failure's representation: be it console output, graphical window or
network
> > socket.
> >
> > A method to the Failure class will be added that will return a suitable
> > string representation of a failure. Failure class itself will provide
> > default implementation that will yield formatted traceback.  This way I
> > could subclass Failure that will yield doctest output instead of
traceback.
> >
> > What would you say?
> 
> your analysis is pretty correct.  However, my idea was to just let
> py.test.skip(someobj) internally take care that str(someobj) provides
> the failure message representation.  Doing that by an internal
> subclass is fine but an implementation detail that should not get
> exposed (Failure classes are not visible directly).

Ah, now I got you, thanks for patience.

Yes, the trick with skip() works fine but, IMO, is no more then a temp hack.
Errors in doctest-based tests are failures and should be reported as such. 

I can try to work out a patch by implementing solution I have described
above. Alternatively, I'm willing to discuss other approaches you or anyone
else has at hand. I'd really like to have doctest support to work smoothly
and (if possible) be incorporated as part of py.test library.

Here is my conftest.py verbatim, in case anyone interested.


import os.path
import py
import sys
from cStringIO import StringIO
import doctest


class Directory(py.test.collect.Directory): 
    def buildname2items(self): 
        d = super(Directory, self).buildname2items()
        for fn in self.fspath.listdir('*.py'): 
            if fn.basename in d or fn.basename == 'conftest.py': 
                continue
            module = None
            try:
                module = fn.pyimport()
            except ImportError, e:
                continue
            finder = doctest.DocTestFinder()
            doctests = finder.find(fn.pyimport())
            doctests = [t for t in doctests if t.examples]
            if doctests:
                module = DoctestModule(fn, parent=self)
                d[fn.basename] = module
        return d

class DoctestModule(py.test.collect.Module): 
    def funcnamefilter(self, name): 
        return True
    def classnamefilter(self, name): 
        return True
    def buildname2items(self): 
        d = {} 
        finder = doctest.DocTestFinder()
        doctests = finder.find(self.obj)
        for test in doctests:
            if test.examples:
                d[test.name] = DoctestItem(test, self)
        return d

class DoctestItem(py.test.Item):
    def __init__(self, test, parent):
        py.test.Item.__init__(self, test.name, parent)
        self.test = test
    def run(self):
        runner = doctest.DocTestRunner()
        out = StringIO()
        writer = lambda msg: out.write(msg)
        failed, tried = runner.run(self.test, out=writer)
        if failed:
            py.test.skip(out.getvalue())
            #raise self.Failed()