[Python-3000] doctests vs. unittests (was Re: pimp; restructuring the standard library)

[Benji York]

Being off topic, I'm just going to do a drive by and urge people that 
are interested in following up to visit the TIP (testing in Python) list 
at http://lists.idyll.org/listinfo/testing-in-python.

Ron Adam wrote:
> I agree with this completely.  Doctests are very useful for getting the 
> basics down and working both while the code is being written.
> 
> After that, unittests are much better for testing edge cases and making 
> sure everything works including the kitchen sink, the pipes to the sink, 
> the quality of water, etc...  ;-)

In the code bases I'm involved in right now, we use doctests almost 
exclusively, including for the "kitchen sink" tests.  We find the slight 
tenancy toward more and better prose in doctests is especially nice when 
trying to discern what exactly some obscure test code is actually trying 
to verify (particularly important when the test fails).

> Currently the unittest test suite runs tests that are in a know place with 
> known name.  There can be modules in a distribution that are completely 
> untested and you would not know unless you manually checked for this.

Most test runners have coverage reporting options for both unit tests 
and doctests.

> There's been a few times where I started writing doctests for a module with 
> less than 100 lines of code and by the time I was done with the doc tests, 
> it became a 500 line or more module.  The actual code then starts to get 
> lost in the file.
> 
> It would be cool if the documents files could also contain the doc tests 
> instead of them being in the source code.

As mentioned later in this thread this is already possible.  Having a 
separate files for them (one of which is usually named README.txt) is 
quite a bit nicer.

If you write your "whole file" doctests in ReST, you can also render 
them to HTML as is done for the packages we put in pypi (here's a short 
example: http://cheeseshop.python.org/pypi/zc.queue/1.1, ReST source at 
http://svn.zope.org/*checkout*/zc.queue/trunk/src/zc/queue/queue.txt).
-- 
Benji York
http://benjiyork.com