[Python-Dev] Purpose of Doctests [Was: Best practices for Enum]

[Steven D'Aprano]

On 20/05/13 23:38, Antoine Pitrou wrote:
> On Mon, 20 May 2013 23:32:10 +1000
> Steven D'Aprano <steve at pearwood.info> wrote:
>> On 20/05/13 20:45, Antoine Pitrou wrote:
>>> On Sat, 18 May 2013 23:41:59 -0700
>>> Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
>>>>
>>>> We should continue to encourage users to make thorough unit tests
>>>> and to leave doctests for documentation.  That said, it should be
>>>> recognized that some testing is better than no testing.  And doctests
>>>> may be attractive in that regard because it is almost effortless to
>>>> cut-and-paste a snippet from the interactive prompt.  That isn't a
>>>> best practice, but it isn't a worst practice either.
>>>
>>> There are other reasons to hate doctest, such as the obnoxious
>>> error reporting.  Having to wade through ten pages of output to find
>>> what went wrong is no fun.
>>
>> Ten pages of broken unit tests are no picnic either.
>
> You didn't understand the objection. I'm talking about *one* broken
> doctest in a sea of non-broken ones. For some reason doctest (or its
> unittest driver) insists on either displaying everything, or nothing.
> It doesn't only print the errors and leave the rest silent.


It sounds like you are inadvertently calling doctest with the verbose option. It is not standard behaviour to display "everything or nothing". Here is the output from 1 failing test out of 112, with absolutely nothing edited.


[steve at ando ~]$ python test.py
**********************************************************************
File "test.py", line 224, in __main__
Failed example:
     len("abcd")
Expected:
     24
Got:
     4
**********************************************************************
1 items had failures:
    1 of 112 in __main__
***Test Failed*** 1 failures.


If I had any criticism of doctest, it would be that by default it prints nothing at all if all tests pass. I hate that, ever since I had a bunch of doctests that for about a week I thought were passing when in fact they weren't running at all. So now I always write something like this:


if __name__ == '__main__':
     import doctest
     failed, tried = doctest.testmod()
     if failed == 0:
         print("Successfully ran %d tests" % tried)



-- 
Steven