[LONG] docstring-driven testing

[zipher]

On Saturday, March 6, 1999 12:00:00 AM UTC-8, Tim Peters wrote:
> If you're like me, you've been using Python since '91, and every scheme
> you've come up with for testing basically sucked.  Some observations:
> 
> + Examples are priceless.
> 
> + Examples that don't work are worse than worthless.
> 
> + Examples that work eventually turn into examples that don't.
> 
> + Docstrings too often don't get written.
> 
> + Docstrings that do get written rarely contain those priceless examples.
> 
> + The rare written docstrings that do contain priceless examples eventually
> turn into rare docstrings with examples that don't work.  I think this one
> may follow from the above ...
> 
> + Module unit tests too often don't get written.
> 
> + The best Python testing gets done in interactive mode, esp. trying
> endcases that almost never make it into a test suite because they're so
> tedious to code up.
> 
> + The endcases that were tested interactively-- but never coded up --also
> fail to work after time.
> 
> About a month ago, I tried something new:  take those priceless interactive
> testing sessions, paste them into docstrings, and write a module to do all
> the rest by magic (find the examples, execute them, and verify they still
> work exactly as advertised).
> 
> Wow -- it turned out to be the only scheme I've ever really liked, and I
> like it a lot!  With almost no extra work beyond what I was doing before,
> tests and docstrings get written now, and I'm certain the docstring examples
> are accurate.  It's also caught an amazing number of formerly-insidious
> buglets in my modules, from accidental changes in endcase behavior, to hasty
> but inconsistent renamings.
> 
> doctest.py is attached, and it's the whole banana.  Give it a try, if you
> like.  After another month or so of ignoring your groundless complaints,
> I'll upload it to the python.org FTP contrib site.  Note that it serves as
> an example of its own use, albeit an artificially strained example.
 
Doctests are so cool, I think it should be integrated into the interpreter environment with a test() built-in to encourage people documenting their code and Test-Driven-Development.  A companion to help().

Oh wait, this thread is 14 years old...

;^)