[Python-ideas] doctest

[Steven D'Aprano]

Nick Coghlan wrote:
> On Sat, Feb 18, 2012 at 7:57 AM, Mark Janssen <dreamingforward at gmail.com> wrote:
>> Anyway... of course patches welcome, yes...  ;^)
> 
> Not really. doctest is for *testing code example in docs*. If you try
> to use it for more than that, it's likely to drive you up the wall,

Really? Not in my experience, although I admit I haven't tried to push the 
envelope too far.

But I haven't had any problem with a literate programming model:

* Use short, self-contained but not necessarily exhaustive examples in the 
code's docstrings (I don't try to give examples of *every* combination of good 
and bad data, special cases, etc. in the docstring).

* Write extensive (ideally exhaustive) examples with explanatory text, in a 
separate text file.

I generally do this to describe, explain and test the interface, rather than 
the implementation, but I see no reason why it wouldn't work for the 
implementation as well. It would require writing for the next maintainer 
rather than for a user of the library.

In the external test text file(s), examples don't necessarily need to be 
self-contained. I have an entire document to create a test environment, if 
necessary, and can include extra functions, stubs, mocks, etc. as needed, 
without clashing with the primary purpose of docstrings to be *documentation* 
first and tests a distant second.

If need be, test infrastructure can go into an external module, to be 
imported, rather than in-place in the doctest file.

In my experience, this works well for algorithmic code that doesn't rely on 
external resources. If my tests require setting up and tearing down resources, 
I stick to unittest which has better setup/teardown support. (It would be hard 
to have *less* support for setup and teardown than doctest.) But otherwise, I 
haven't run into any problems with doctest other than the perennial "oops, I 
forgot to escape my backslashes!".



-- 
Steven