[SciPy-user] For the testing BOF...

[Fernando Perez]

Hi all,

this is something I just put out to the ipython-dev list.  After I
sent out the proposal for the testing BOF, I decided to actually write
something on testing so I could dive into a big refactoring I have
here at work with better tools.  This is the result, and I'll probably
use it as a starting point for some of the discussion next week.  It's
pure alpha code and written with a very specific purpose:

- make writing tests work with minimal friction in *my* workflow,
while using only unittest and doctest for actual test construction.

So it is NOT a nose/py.test/xxx competitor, nor is it intended to ever
be one.  It's just a bit of support code to ease the integration of
tests while developing as painless as possible.

If anyone has interest and has a look between now and next week,
great.  We can discuss it further there.

Cheers,

f

---------- Forwarded message ----------
From: Fernando Perez <fperez.net at gmail.com>
Date: Aug 8, 2007 5:39 PM
Subject: ANN: SnakeOil - Python testing that doesn't squeak :)
To: IPython-dev List <ipython-dev at scipy.net>


Hi all,

For a long time I've been really fed up with feeling that the workflow
for testing in Python just isn't very fluid.  Generating doctests
requires typing code in text mode, rerunning and modifying them is
annoying, having a standalone test script become a unit test isn't
convenient, and the worst of all, writing parametrized tests in
unittest is a huge PITA.

IPython SVN now has a doctest profile to help a little (and I'll add a
magic to switch at runtime in and out of it, hopefully later tonight).
 But that's just a fix for one small annoyance among many.

I know there are testing frameworks out there that address some of
these issues (nose, py.test), but I wanted something small(ish), that
could be used without installing anything (easy to copy inside the
test area of any project and carry it from there, or install it in a
private area after a rename, etc), and that would only depend on the
stdlib (+ ipython, of course, since I use that everywhere).

So the IPython team is happy to announce SnakeOil:

svn co http://ipython.scipy.org/svn/ipython/ipython/branches/saw/sandbox/snakeoil/snakeoil

It's raw alpha, svn-only, lightly tested (about 2k LOC).  But it lets
me do *easily* all the things I needed to feel that adding tests
wasn't getting in *my* way.  If it doesn't work for you, no worries,
you can get your money back.

It's also fairly hackish in places (surprise, coming from me).
Unittest doesn't lend itself to much of anything, really, so I had to
beat it into behaving in a few places.  Still, I wanted to stick to
valid unittest classes so that using SnakeOil would not make your
tests citizens of yet-another-framework.

The doc/ directory has a reST document with some more details as well
as a fully worked set of examples.

The highlights:

- Easy creation of parametric tests (unittests that take arguments).
That includes tests that share state (yes, I know about shared state
in tests.  I'm an adult, I know what I'm doing and I need that.  Now
unittest, get out of my way and let me actually work).

- Immediate use of any standalone testing script as a unit test, without having
to subclass anything.

- Easy mechanisms for creating valid doctest (.txt) files from true Python
sources, so that one can edit real Python code in an editor and convert that
code to a set of doctests with minimal effort.



This will be integrated in ipython1 later, but for now it sits in the
sandbox.  We'll announce when we move it out.  We first need to check
that everything works with Twisted trial correctly (I think it does,
but I haven't checked enough yet).

Comments/feedback welcome.

Cheers,

f