[py-dev] core design of py.test broken?

[Armin Rigo]

Hi Holger,

On Wed, Jan 05, 2005 at 12:49:29AM +0100, holger krekel wrote:
> Then we also execute test methods across process boundaries 
> (with --session and lately --exec) and across python versions
> and platforms (and soon across networks).  And we want to
> "memorize" especially failing tests across those barriers

When reading py/test/item.py, I feel uneasy about this global object
SetupItem.state which encapsulates the notion of which collectors and items
are currently "up".  This is a bit strange because there is another parallel
mecanism that looks cleaner, which is already used for reporting: naively, it
seems that it would just work if a collector or item were set up when the
method Driver.run_collector_or_item() starts, and teared down when it ends.  
This would work independently of any notion of extpy path to name each item.

Note that this is only about setup/teardown.  For purposes like --session or
remote execution of tests, we clearly need a persistent notion of "name" or
"path" of test items:

> Basically we could think of a Collector offering two concepts: 
> 
>     listdir()   # list all the possible Items (test methods) and 
>                 # "recursion" points, i.e. Collectors, each 
>                 # of which has a specific associated name
> 
>     join(name)  # get a contained Collector/Item specified 
>                 # by such a name  

In this model generative tests are themselves collectors, not items, and the
listdir() should enumerate all the sub-tests by enumerating the user-defined
generator.  The easier would be to use integer indices as the name of the
sub-items (the first one get the name 0, the next one the name 1, and so on).  
It means that we shouldn't add or remove sub-tests in the middle of the list
if we are e.g. running in --session, otherwise py.test will mess up the
various sub-tests...  but I don't see any obvious workaround.  In general I 
think that using just lists of names or numbers as paths to individual test 
items looks good.

The semantics of setup/teardown could then be cleanly integrated in this 
model, by putting the related knowledge on the individual Collector classes, 
e.g. with two new methods on all collectors and items: enter() and leave().  
The Module collector would call setup_module() in its enter() method, etc.


A bientot,

Armin.