[Python-Dev] Community buildbots and Python release quality metrics

[Georg Brandl]

glyph at divmod.com schrieb:
> On 03:42 pm, ncoghlan at gmail.com wrote:
>>glyph at divmod.com wrote:
> 
>>beta 1 has some trouble running *our* test suite - I'd be fairly 
>>surprised if the community buildbots were in significantly better 
>>shape.
> 
> That's another problem, yes :)
>>>The community buildbots have been in a broken state for months now[1].
> 
>>If the community buildbots aren't largely green by the time beta 2 
>>comes out, that's when I'll agree we have a problem - they should 
>>definitely be green by the time first release candidate comes out.
> 
> I have mixed feelings about this: on the one hand, if this were a matter 
> of public record, I would have known that this was too early to start 
> complaining, and we could have saved everyone a lot of time reading 
> these messages.  That would be good; I would prefer to just let everyone 
> work without bothering about process.

The problem is that not enough people care about the community buildbots,
and this isn't likely to improve since we have not enough manpower to do
it.

> On the other hand, it's much easier to deal with test failures as they 
> arise than in one giant chunk of work at the end of the development 
> process.  I spoke to a few core developers at PyCon who thought that the 
> buildbots should always be green and any change that broke them should 
> be reverted.  They may remain nameless unless they wish to raise their 
> hands :-) but that's definitely how I feel about it.

I think no core developer will not tell you that the (core) buildbots should be
green all the time. As for reverting changes that break, I'd support this
only for changes that break *all* of them. For example, I only use one
platform to develop on (and I guess it's the same for many others), having
the buildbots go red on another platform means I can try to fix the issue.

Georg