[pypy-dev] Improved Unit Test Framework

[Laura Creighton]

I couldn't remember what Peter had told me about his testing framework,
so I asked him.  Now he is curious as to what we want to do differently.

Laura

------- Forwarded Message

Return-Path: peter at engcorp.com
To: Laura Creighton <lac at strakt.com>
Subject: Re: Unit testing
References: <200310171413.h9HEDlLH014913 at ratthing-b246.strakt.com>

Laura Creighton wrote:
> 
> DO I remember it correctly that you have extensively hacked up the
> unittest module to build a nicer framework?  and if so, is what you
> did freely available?  PyPy wants a nicer framework ...

Actually, we carefully avoided touching "unittest.py" at all, as we
were new to this game.  Instead we've added a wrapper ('ktest') around it 
which provides a few features we find useful (but only under Win/DOS).

- - type "unit xxxx" to run unit tests for module xxxx, expecting to find
  them at ./unit/xxxx_unit.py

- - "unit -a" runs all tests in all ./unit subdirectories, recursively

- - "unit -r" randomizes the order of the individual TestCases (but not
  yet the individual test methods) to help catch test-order dependencies

- - all output from tests go to log files as xxxx_unit.log for later
  inspection

- - no graphical abilities whatsoever; no pretty green/red bar (I regret the
  latter part)

- - automatic inclusion of .pth files found in ./unit subfolders, and
  automatic inclusion of parent folder in the sys.path (all tests run
  with the ./unit folder as current directory)

- - automatically use Python 2.0 or 2.2 based on flag file in ./unit folder
  as we have not yet migrated entirely from 2.0

- - probably a few other bits

I seriously doubt that what we have would be of much use to anyone
as they would likely have to adopt a completely different approach to
testing than they currently use. 

The area where we have actually excelled, instead of where we've stuck
to the entirely non-fancy console/DOS interface with limited features,
is in our auxiliary test frameworks.  

One of them, PyJTF, has just been put up on Sourceforge 
(http://pyjtf.sourceforge.net/).  It allows test-driven development 
of Javascript code for the IE browser (again, limited to Win32 :-( ) 
using Python and COM and Twisted to great effect.

The other, as yet unreleased, incorporates a Python-based HC12 CPU
simulator to allow test-driven development of embedded system code
in C.  Tests in Python, code in C, fairly simple interface and 
amazing performance (one set of 98 tests takes only 4 seconds to run).
As this one matures, I hope to release it as well.

If the kunit features are remotely interesting, I can forward the
code and its own tests.

Feel free to post any part or all of this to the PyPy mailing list,
or elsewhere, as you see fit. :-)

- -Peter Hansen

------- End of Forwarded Message