[Python-Dev] Community buildbots and Python release quality metrics

[Guido van Rossum]

On Thu, Jun 26, 2008 at 5:33 PM,  <glyph at divmod.com> wrote:
> OK, fair enough.  Taking a step back, I was pushing this really hard because
> to *me*, it seems like dealing with the influx of bug reports after the fact
> is an unreasonable amount of additional work, whereas immediate reverts are
> just an occasional, temporary inconvenience. Based on my experience, "We'll
> deal with the reports as they come in" sounds like "no".

No, that's not how it was intended. I'd rather hear you tell us about
your pain than telling us how to fix it.

> I think since the last time we had a similar conversation, other Twisted
> developers have been fairly diligent in reporting bugs.  (In the time
> they've been reporting Python bugs, I've mostly been planning a wedding.
> Others who have survived it tell me that eventually, this process ends...
> and then I should be participating more directly.)  I'll try to step up that
> feeback loop and get other projects involved.  I hope I am wrong about
> generating an unreasonable amount of work.

Again, I'd much rather see you generate a huge pile of work for us in
the form of specific bug reports than trying to tell us how to change
our process.

> Thanks.  I appreciate it; an increased emphasis on 3rd party code being
> broken is really what I was looking for.  This is fine with me.  I mean,
> whatever your decision is, I'm going to have to live with it :), but in
> either case, we have to be looking for bugs and helping to investigate them.
>  On my end of things I guess it's not going to make much difference.

Right. The beta period is the time to pay attention to 3rd party
breakage. While we won't make any specific promises, we're taking 3rd
party code breakage very seriously and, depending on the specifics, it
can hold up a release. It just won't be automatic; many 3rd party
breakages are most easily fixed by making small adjustments to the 3rd
party code involved. (Imagine a 3rd party package still using string
exceptions.)

.
.
.

On Thu, Jun 26, 2008 at 6:13 PM,  <glyph at divmod.com> wrote:
> For the record, "automatic revert" does not mean "drop all other work". The
> changeset's still there.  It's still in the revision history.  It can easily
> be applied to anybody's working copy.  It can easily be put into a branch.
>  It can be automatically re-merged later.  I do all of these things all the
> time, and I was *not* intending to suggest that any 3rd-party breakage
> should cause a code freeze or anything even remotely like that.

Still, a revert often holds up other work that depends on the reverted
change. I expect that in most cases, finding the problem and applying
a fix is much less of a burden for the core developer team as a whole
than rolling back, working on a fix, and re-applying. I also expect
that the policy of semi-automated merges from 2.6 into 3.0 would make
a revert even more work, and more likely to cause second-order problem
in the 3.0 branch.

>>> (...) it would have been easier to
>>> convince a Twisted developer to do the work it to keep the community
>>> buildbot green rather than to make it a bit less red.
>>
>> Why? That sounds like it's a problem with the psychology of the
>> Twisted developers.
>
> I hardly think it's unique to us.  TDD test runners typically only know 2
> colors and 2 states: "passed" and "fails".  Once you're in the "fail" state,
> you tend to accumulate more failures; there's a much bigger bang for your
> buck.  Better tools with nicer interfaces would let you easily mark
> individual tests as "usually intermittent" or something and make it a
> "slightly dirty green" or something, but we don't have those.  The move from
> "red" to "green" is much more psychologically significant to just about
> anyone I know than the move from "red but 14 failures" to "red but 12
> failures".

Still sounds like perhaps you're too much focused on the "green bar".
I know this is encouraged by XP, but I don't want to have a religious
debates about whether XP is the right development model for Python.
(The time would be better spent on improving the buildbot reporting!)

> Despite the idea's origins in a now-highly-controversial book on
> criminology, this is often referred to in XP discussion circles (wikis and
> the like) as the "fix broken windows" collaboration pattern.  I notice this
> in lots of other areas of my life; a little pile of papers tends to become a
> big pile of papers, the first dent in a car makes the driver a little less
> careful, and so on.

For me, with that first dent comes the realization that it's really
not worth it to obsess over scratches, and that makes me a more
relaxed and hence *better* (because less stressed) driver. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)