[Distutils] RFC: Standard Declaration of tests in eggs

[Jim Fulton]

David Fraser wrote:
> Jim Fulton wrote:
>> Here is a rough draft proposal for declaring tests in eggs:
>>
>> Introduction
>> ============
>>
>> Software packages should have automated tests.  Consumers of
>> packages will often want to run these tests.  Tools should be able to
>> do this automatically.  This proposal seeks to provide a way for
>> automated tools to discover tests in distributions, including eggs, so
>> that tests can be run or so that test runners can be automatically
>> created to run the tests.
>>
>> Proposal
>> ========
>>
>> This proposal aims to be extremely simple.  It has 2 parts:
>>
>> 1. A 'test_suite' entry point is defined.  An egg can provide zero or
>>     more test_suite entry points.  These entry points will define
>>     callable objects that can be called without arguments and that
>>     return unittest test suites.
>>   
> How would this work if for example, you're using an alternative testing 
> framework (like py.test) for your test?

I'm not familiar with py.test.  I guess it's not based on unittest.
Why?  Couldn't it at least have a unittest wrapper, like the one I
wrote for doctest?

Even though I use doctest almost exclusively, I view unittest
as a common API that various testing frameworks can and should
play with.  I certainly think there should be some common API
like that and see unittest as the incumbant.

 > It would be nice to be able to bootstrap it :-)

What do you mean by that?

>> 2. An optional 'tests' extra is defined.  When creating test runners
>>     or dynamically loading distributions to load tests, any
>>     distributions listed in extra requires for the 'tests' extra shall
>>     be included in the working set for the test runner.
>>   
> Great, so at least the testing framework could be declared as a dependency

My assumption (and my use case) is that you'd run whatever test runner
you want and tell it to run tests from some given eggs.  So the test runner
would already be loaded.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org