[Python-Dev] Python Language Summit at PyCon: Agenda

[Robert Collins]

On 5 March 2013 05:34, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw <barry at python.org> wrote:
>>
>> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
>>
>> >It is of course possible for subunit and related tools to run their
>> >own implementation, but it seems ideal to me to have a common API
>> >which regular unittest, nose, py.test and others can all agree on and
>> >use : better reuse for pretty printers, GUI displays and the like
>> >depend on some common API.
>>
>> And One True Way of invoking and/or discovering how to invoke, a package's
>> test suite.
>
>
> How does unittest's test discovery not solve that?

Three reasons
 a) There are some bugs (all filed I think) - I intend to hack on
these in the near future - that prevent discovery working at all for
some use cases.
 b) discovery requires magic parameters that are project specific
(e.g. is it 'discover .' or 'discover . lib' to run it). This is
arguably a setup.py/packaging entrypoint issue.
 c) Test suites written for e.g. Twisted, or nose, or other
non-stdunit-runner-compatible test runners will fail to execute even
when discovered correctly.

There are ways to solve this without addressing a/b/c - just defining
a standard command to run that signals success/failure with it's exit
code. Packages can export a particular flavour of that in their
setup.py if they have exceptional needs, and do nothing in the common
case. That doesn't solve 'how to introspect a package test suite' but
for distro packagers - and large scale CI integration - that doesn't
matter.

For instance testrepository offers a setuptools extension to let it be
used trivially, I believe nose does something similar.

Having something that would let *any* test suite spit out folk's
favourite test protocol de jour would be brilliant of course :).
[junit-xml, subunit, TAP, ...]

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services