[Python-Dev] Community buildbots and Python release quality metrics

[glyph at divmod.com]

On 07:44 pm, g.brandl at gmx.net wrote:
>At no time will a policy "the community buildbots must be green" be
>useful: the tests that run on these buildbots are not under our 
>control,
>so if the tests test things we deem non-public we can't do anything
>about it. (And we may have a hard time convincing project authors to
>change the tests if we promised to make them run anyway.)

That's not what I'm suggesting.

If there is a legitimate disagreement between Python developers and 
developers of a project about whether an API should continue to be 
supported, then clearly, the Python developers get to win.  Welcome to 
open source.

However, I believe that the core team is not paying attention to other 
projects breaking.  I'm not saying that you want to make breaking 
changes, or that bug reports are not dealt with, but the problem is that 
dealing with these problems after the fact makes it _much_ more painful. 
When you get to the release, and you have 30 bug reports due to other 
projects breaking, they get triaged, some get left in, and some features 
of lots of different projects are left broken.  And many projects do not 
bother to test with the beta.

If developers were simply presented with the results of their changes 
immediately ("you broke twisted, django doesn't import, zope segfaults 
and mercurial destroys your repository") then perhaps they would weigh 
the benefits of compatibility differently, and do the trivial, obvious 
fix immediately, rather than waiting 6 months and having to figure out 
what the heck change caused the bug.  I accept the fact that python core 
development is done differently than i.e. Java development, and some 
backward compatibility may simply be broken.

Case in point: changes to the warnings module being discussed in a 
separate thread break a significant number of Twisted's tests, because 
they remove functionality which is not public but is required to test 
code that uses the warnings module.  Would Brett have taken this into 
account if he knew about it immediately when revision 62303 was 
committed?  Maybe not, but now that the code is written and done, 
there's significantly less motivation to go back and fix it.  At the 
time it might have only been a few minutes' work to continue supporting 
the use-case which Twisted needs.  Brett wouldn't even have necessarily 
needed to do the work: it would have been easier to convince a Twisted 
developer to do the work it to keep the community buildbot green rather 
than to make it a bit less red.  Now, though, it's much easier to say 
"oh well, that's private, you lose".  I don't ascribe this to malice - 
it really *would* be much harder to fix it now, for us as well as for 
him.