[py-dev] Re: [Distutils] setuptools presentation

[Ian Bicking]

holger krekel wrote:
>>Well, I don't know what exactly my opinion is.  At a py.test user I 
>>don't have TestSuites for my tests.  I've argued py.test should load 
>>TestCase-based tests by default, but this is kind of the opposite.  I 
>>don't think it would be that hard to produce such suites; the test items 
>>that py.test collects could just be stuffed into a TestCase.
> 
> 
> I agree without having looked deeply into it.  Providing TestSuite
> instances that adapt running tests in a simple way with part 
> of the py.test machinery should be quite feasible.  Would probably
> mean that some of py.test's execution features would be switched
> off (like stdout/stderr capturing). 
> 
> However, as Ian hints at, py.test itself is already a project independent
> entry point for running tests in a given directory or for a given project. 
> 
> 
>>The result would be acceptable as a sort of "this package thinks it runs 
>>okay" test.  It's not the frontend I'd like to give to users.
> 
> 
> unsurprisingly, i agree :-)

Actually I mistyped -- for developers (or potential developers) I think 
py.test's full featureset and frontend should be available.  However, 
for mere library users there's a use case where they want to simply know 
that the tests pass, that the package is ok; failures are not expected. 
  At least, when I was talking about this with some Perl people, they 
specifically said they like that CPAN automatically runs tests when you 
download a package.

I think a uniform command-line interface is nice in this case; the user 
is just look for "ok" or "not ok", they aren't looking for a development 
tool.  And it's possible that "setup.py test" should be that uniform 
interface.  That doesn't exclude the possibility of a separate command 
like "setup.py pytest".

>>I suppose a package could add an entry point that overrides
>>the normal setuptools test command...?
> 
> Yes, i guess something needs to be configurable there as far as i
> understand the situation.  I also presume that "setup.py test"
> would allow a custom test method to actually perform the execution
> of tests, not just provide a TestSuite.  
> 
> If so, it should at best become a simple matter of how a
> package can signal to setuptools that it wants its tests to be
> handled by py.test in which case the work would be defered to
> the (neccessarily) installed py library when "setup.py test"
> is invoked.  It shouldn't be required from each application
> to provide this glue code as it should be generic. 
> 
> actually, how does "setup.py test" work today and which
> applications/projects are integrated with it if i may ask? 

AFAIK no applications are integrated.  Well, maybe PEAK... this is all 
rather new.  It's documented here:

http://peak.telecommunity.com/DevCenter/setuptools#test-build-package-and-run-a-unittest-suite

However, I believe each command to setup.py is simply an "entry_point". 
  I can't find documentation for that at the moment.  But it's something 
like this in the setup.py file:

   entry_points={
     'distutils.commands': [
       'test=test_module:test_command',
     ],
   }

And then that test_command object (a class) has a specific interface 
(based on the distutils.Command class).  So potentially a py.test-using 
project could do:

   entry_points={
     'distutils.commands': [
       'test=mypkg.commands:test']}

   # in mypkg.commands:
   from py.test.disutils import test


I think that would override "setup.py test" for just that package.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org