[Python-Dev] Python Language Summit at PyCon: Agenda

[Robert Collins]

On 4 March 2013 18:54, Guido van Rossum <guido at python.org> wrote:
> On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins
> <robertc at robertcollins.net> wrote:
>> I'd like to talk about overhauling - not tweaking, overhauling - the
>> standard library testing facilities.
>
> That seems like too big a topic and too vague a description to discuss
> usefully. Perhaps you have a specific proposal? Or at least just a use
> case that's poorly covered?

I have both - I have a draft implementation for a new test result API
(and forwards and backwards compat code etc), and use cases that drive
it. I started a thread here -
http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html
, with blog posts
https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/
https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/
https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/
https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/

They are focused on subunit, but much of subunit's friction has been
due to issues encountered from the stdlibrary TestResult API - in
particular three things:
 - the single-active-test model that the current API (or at least
implementation) has.
 - the expectation that all test outcomes will originate from the same
interpreter (or something with a live traceback object)
 - the inability to supply details about errors other than the exception

All of which start to bite rather deep when working on massively
parallel test environments.

It is of course possible for subunit and related tools to run their
own implementation, but it seems ideal to me to have a common API
which regular unittest, nose, py.test and others can all agree on and
use : better reuse for pretty printers, GUI displays and the like
depend on some common API.

> TBH, your choice of words is ambiguous -- are you interested in
> overhauling the facilities for testing *of* the standard library (i.e.
> the 'test' package), or the testing facilities *provided by* the
> standard library (i.e. the unittest module)?

Sorry! Testing facilities provided by the standard library. They
should naturally facilitate testing of the standard library too.

-Rob

> --
> --Guido van Rossum (python.org/~guido)



-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services