[Python-Dev] Community buildbots (was Re: User's complaints)

[Nick Coghlan]

Jeremy Hylton wrote:
>> When the slave suffers a real failure due to a backwards 
>> incompatibility, it
>> will take a developer of the application to figure out what it was 
>> that broke
>> the application's tests.
>>
>> So while I think it's a great idea, I also think it will need significant
>> support from the application developers in debugging any buildbot 
>> failures to
>> really make it work.
> 
> These buildbots should run the tests from stable, released versions of
> external packages, assuming that those packages don't ship releases
> with failing tests.  If you ran the test suite for a Zope release and
> had a test failure, I think there would be a reasonable expectation
> that it was a Python bug.

Definitely, but there's a difference between "bug that broke Python's own unit 
tests" and "change to a behaviour that package X depended on".

It's the latter cases that the external buildbots would be looking for - and 
while some of those will be shallow enough that the regression is obvious from 
the unit test error message and the recent Python checkins, the non-obvious 
ones will require a collaborative resolution.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org