[Python-Dev] built-in Python test runner (was: Python Language Summit at PyCon: Agenda)

[Daniel Holth]

On Tue, Mar 5, 2013 at 2:49 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote:
>
>>Doesn't setuptools/distribute already have a setup.py test command? That
>>seems like the easiest way forward?
>
> Yes, and in theory it can make `python setup.py test` work well.  But there
> are lots of little details (such as API differences for ensuring that doctests
> run, "additional tests" discovery, etc.) that make this often not work so well
> in practice.  Some of that is social and some of it is technical.  I still
> claim that including test suite information in a package's metadata would be a
> win, but maybe that's just too much to hope for right now.

It would be a win, but "parsing the metadata" is just not what happens
right now, let alone writing anything about which and where the
modules are defined in the sdist. We can barely install packages by
using the dependency metadata from PKG-INFO; pip always re-generates
it from "setup.py egg_info".

Your testing metadata prototype would only have to write two lines to
the metadata instead of one a-la: Extension: flufl; flufl/test_suite:
nose.collector; document the extension; write some tool to actually
parse the metadata and invoke the tests; it may become a core feature
in the next version, or having a monolithic specification may become
less important.

Thanks,

Daniel Holth