[py-dev] running doctest-based tests

[David Stanek]

On Sat, Jun 11, 2005 at 02:02:19PM +0300, Max Ischenko wrote:
> Holger,
> 
> 
> > I also think that your idea of just copying and maybe slightly modifying
> > the 2.4 doctest module into the py lib makes sense. The general idea is
> > that py.test and the py lib should work sanely on python 2.2, 2.3, 2.4
> > and 2.5/cvs.
> 
> Well, I didn't mean py lib specifically, just a casual user who wants to use
> a doctest module. But including it in a py lib may be a sane idea. Module is
> in the public domain by the way.
>  
> > If we integrate doctest support natively, then i now think that we
> > should aim for offering simple conftest.py configuration items like:
> 
> If we integrate doctest support natively do we need conftest.py in the first
> place?
> 
> Currently you need no conftest.py to collect and run test code. 
> 
> Why don't integrate doctests in the same way? Guess a performance hit may be
> an issue but then a switch can be added that will disable (or enable)
> doctests support.
> 
> Just an idea.
> 
> >     test_files = ['test_foo.py', 'test_bar.py']
> >     extra_doctest_files = ['whatever.py', ...]
> > 
> > and test files should also be searched for doctests, btw. There would
> > then be no need to write custom Collectors/Items for changing the order
> > or the list of to-be-examined doctest modules.  The above example
> > file lists would only be associated with the directory containing
> > the conftest.py file and not any subdirectories.  On the implementation
> > side, the given files should always be imported via the path.pyimport()
> > method which imports modules with their fully qualified name.
> > (http://codespeak.net/py/current/doc/test.html#package-name)
> 
> The situation is slightly complicated by the fact that there are two options
> to specify doctests (inline or in a separate file) which will require
> different treatment by the library.
> 
> At a bare minimum, I'd be happy if py.test just provided this boring code
> (DoctestItem, DoctestModule, etc) that is needed to collect and execute
> tests. If this is available, I can create short and sweet conftest.py to
> build correct configuration for my exact situation.
> 
> At the higher level, py.test may support some syntax sugar in conftest.py
> (like the one you described) that will aid writing this configuration file.
> 
> Going even further, py.test may be able to collect and run doctests
> directly, without the need for user to write any arcane configuration files.
> Of course, for a typical scenario, just like it does with normal tests. If
> user has custom requirements and non-standard file hierarchy s/he will need
> to write a conftest.py.
> 
> IOW, I'd like that "native doctests support" will mean that doctests become
> first-class citizens, just like normal tests are.
> 
> That's my opinion anyway. For a start I just want doctest failures be
> reported properly. ;)
> 
> > David, are you interested in taking a stab at this?  It's a bit
> > involved, i am afraid, also because it involves changing some reporting
> > functionality.  If you or anyone else would like to do it, please make
> > sure to write proper tests for any new features even though it's sometimes
> > a bit involved to do that.  I sometimes spend half the time for writing
> > tests for new py lib features but it really pays off.
> 
> Guess I should wait a bit and see what David will produce.
> 
> David, if you need my help - just ask.

I am working integrating doctests into the library so that a
conttest.py will not bee needed. I had talked to Holger a couple of
days ago about adding a --doctests option to include collecting
doctests and executing them. I have a bunch of patches that are a
starting point. I will be putting all of the changes in their own
branch, so that I will not affect the release schedule. Changes will
start showing up here:
  http://codespeak.net/svn/py/branch/dist-doctest

Max,
Once I do start commiting them, I'll post a request for comments. I
think I will take you up on your offer. Once I commit existing
changes we can chat a little about direction. Do you use IRC, AIM,
or some other type of live medium?

-- 
David Stanek
www.roninds.net

GPG keyID #6272EDAF on http://pgp.mit.edu
Key fingerprint = 8BAA 7E11 8856 E148 6833  655A 92E2 3E00 6272 EDAF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20050611/77c45ad6/attachment.pgp>