[py-dev] running doctest-based tests

[holger krekel]

Hi David, hi all, 

On Fri, Jun 10, 2005 at 08:54 -0400, David Stanek wrote:
> On Fri, Jun 10, 2005 at 01:22:00PM +0200, holger krekel wrote:
> > Hi Max, hi David, hi all, 
> > 
> > Oh sure, i am sorry.  Of course, i should have talked about 
> > py.test.fail() all along! skip() doesn't make any sense here. 
> > 
> > I also think that your idea of just copying and maybe slightly modifying 
> > the 2.4 doctest module into the py lib makes sense. The general idea is
> > that py.test and the py lib should work sanely on python 2.2, 2.3, 2.4 
> > and 2.5/cvs. 
> > 
> > If we integrate doctest support natively, then i now think that we 
> > should aim for offering simple conftest.py configuration items like: 
> > 
> >     test_files = ['test_foo.py', 'test_bar.py']
> >     extra_doctest_files = ['whatever.py', ...]
> > 
> > and test files should also be searched for doctests, btw. There would 
> > then be no need to write custom Collectors/Items for changing the order 
> > or the list of to-be-examined doctest modules.  

> Holger,
> After we talked last night I was thinking about making the change to
> both py.test.skip() and py.test.fail(). I agree with Max that this
> is a hack and probably not the best way to do it.

It really depends on the purpose.  For some purposes like the
custom doctest-link checking support, having py.test.fail() 
cause the sesssion to display brief information without 
a traceback seems to make perfect sense.  For doctests 
we need more support than just this, though. 

> So I started to work in a slightly different direction. I
> was thinking a more general approach would be to factor out
> the view logic into it's own class heirarchy.

Just so you know: when i hear the word "class hierarchy" i usually
get afraid :-) 

> Here is what I was thinking:
>  * added an additional param to fail() and skip(), defaulted to None
>  * factored out the py.test.TerminalSession's repr_* functions into
>    py.test.view.(BaseOutcomeView & DefaultOutcomeView) classes

If you mean that py.test should grow a new 'view' sub-namespace 
then i am not convinced yet. I do think that separating out 
representation/view logic some more from the sessions makes
sense, though. 

> Then the conftest.py can select a custom view class and attach it to
> the outcome.

Where exactly is the need for that?  For adding proper doctest 
support we don't need to expose new names.  py.test should 
just internally make sure it does nicely collect, run and 
report doctests. 

David, to allow you and others to just head off into the 
direction you are imagining, just do a 

    svn cp http://codespeak.net/svn/py/dist \
           http://codespeak.net/svn/py/branch/dist-doctest 

and work in that branch.  I have a feeling that you
could otherwise not work incrementally and get early 
feedback (until py is released the 'dist' branch should
remain pretty stable).  Tested Code sometimes says 
more than words on mailing lists :-) 

cheers, 

    holger