[py-dev] utest thoughts

[holger krekel]

Hi Ian, 

i have a second thought regarding one of your suggestions/comment 
regarding custom test collectors ... 

[Ian]
> [me]
> >b) collectors always yield collectors or Units, not plain functions 
> >   so you would actually do 
> >
> >      yield py.test.Unit(my_custom_test_function, *args)
> >
> >In the current py code (see my other postings) you'll find 
> >in py/test/test/data/Collector.py the current way to do custom 
> >collectors.  This Collector class is instantiated with 
> >the module's location and takes over the collecting 
> >process for the module.  No further attempt is made 
> >to collect anything from a module which contains a 
> >custom "Collector" object. 
> 
> I have a problem with APIs that jump in complexity.  In this case you 
> can forget about Unit until you start needing to generate multiple tests 
> programmatically -- your functions *were* your units, until that point. 
>  If functions are units in one place, they should be units everyplace.

I agree that a jump in complexity needs to be justified and 
for the discussed case it's arguable: Custom collectors 
or generating custom tests should be easy and if possible a 
"no-API" thing. 

Today i thought that maybe it isn't such a bad idea to allow 
generative tests which would come close to what collectors provide. 

    def test_generating(self): 
        for x in someiter:
            yield x 

would yield new test methods/objects which would be wrapped in an 
appropriate Item-class in order to allow the reporter to uniformly 
access information about test items. 

The py.test.Driver could detect "generating tests" by checking
if ``test_obj()`` returns a generator or if ``iter(test_obj)``
succeeds.  

This, however, would defer the decision if something is a collector 
aka a "generative test" or a plain test item to a very late point 
which leads to the the question if the distinction between collectors 
and test items can't be removed altogether. I think i am going to 
give this a try soon. Comments welcome. 

cheers, 

    holger