[Python-Dev] Community buildbots and Python release quality metrics

[Georg Brandl]

glyph at divmod.com schrieb:

> Another way to phrase this question is, "whose responsibility is it to 
> make Python 2.5 programs run on Python 2.6"?  Or, "what happens when the 
> core team finds out that a change they have made has broken some python 
> software 'in the wild'"?
> 
> Here are a couple of ways to answer these questions:
> 
>   1) It's the python core team's responsibility.  There should be Python 
> buildbots for lots of different projects and they should all be green 
> before the code can be considered [one of: allowed in trunk, alpha, 
> beta, RC].
>   2) It's the python core team's responsibility as long as the major 
> projects are using public APIs.  The code should not be considered 
> [alpha, beta, RC] if there are any known breakages in public APIs.
 >
>   3) It's only the python core team's responsibility to notify projects 
> of incompatible changes of which they are aware, which may occur in 
> public or private APIs.  Major projects with breakages will be given a 
> grace period before a [beta, RC, final] to release a forward-compatible 
> version.
>   4) The status quo: every new Python release can (and will, at least 
> speaking in terms of 2.6) break lots of popular software, with no 
> warning aside from the beta period.  There are no guarantees.

Python's backwards compatibility has never extended to non-public APIs.
That rules out 1). 2) would be nice, but to be honest, 2) and 3) are
unrealistic given the number of Python core developers and the tasks
already at hand. 4) is not acceptable either.

However, you seem to be overlooking that there's a 3.5) in there: it's
the Python core team's responsibility as long as the major projects are
using public APIs; but to reduce the workload every project should watch
its own buildbots and notify the core team using the issue tracker in
order to get the issue resolved.

At no time will a policy "the community buildbots must be green" be
useful: the tests that run on these buildbots are not under our control,
so if the tests test things we deem non-public we can't do anything
about it. (And we may have a hard time convincing project authors to
change the tests if we promised to make them run anyway.)

Georg