[Python-Dev] Trimming the fat from "make quicktest" (was Re: I am now lost - committed, pulled, merged, what is "collapse"?)

[Stephen J. Turnbull]

Nick Coghlan writes:

 > (i.e. less than a minute) because you're going to be running it three
 > times in quick succession (perhaps 3 if it applies to 2.7 as well).

Nobody says it's got to be *you* that runs the tests, just that they
need to be run before pushing to public repo.

Here's a simple way to deal with this: have two repo URLs,
"pull-source" and "push-target".  Under (currently) normal commit
pace, "pull-source" and "push-target" resolve to the same physical
repo, the other repo(s) sync(s) to that one in the background (or they
could be cloned at need, in advance of the sprint).  Sprinters clone
from "push-target", non-sprinters from "pull-source".

When sprinting, change "pull-source" to resolve to a mirror, and close
it to commits, protecting the non-sprinters from instability.  Let
the sprinters have at it, and start a test process walking the commits
in "push-target", testing each one.  As each commit is validated,
merge it to a third repo "validated" (could be a named branch, but I
suspect named branches would suck for this purpose, because eventually
you want these commits on the default branch or the appropriate named
branch).

So far, so good.  Sprinters don't need to reconfigure their
workspaces, they don't need to run the full test suite before merging.
The main issue remaining is what if the tests fail?  Now you have to
fix the commit and do a merge dance.  I'm not sure what the best way
to do this is.

My thinking if merges are used is that you *need* to

1. rewind the workspace to the busted commit or just before it (ie,
   keeping the changes for the busted commit)
2. fix it
3. commit, creating a new head
4. rebase children of the busted commit on the new head (*without*
   destroying the original branch
5. merge the original branch into the rebased branch
6. point the test process at the new tip

Rationale:

1. The test process has not tested children of the busted commit, you
   can't just commit the fix on top -- it will most likely fail the
   same way for each child until reaching the fix.
3. See 1.
4. See 1 for "rebase".  You want to keep the original branch because
   other sprinters' workspace configs know about it, and will get very
   confused if it disappears from the push-target repo.
5. This forces other sprinters to update before pushing, thus
   incorporating the fix in their work, and getting them back on the
   same page as the test process.
6. Commits don't know their children, so you'll have to reinitialize
   the test process to walk the branch backward.

I think the process 1-6 is excessively complex for most contributors,
and the rebase itself risks conflicts.  So I'm not entirely happy with
this.

Another possibility would be to cherrypick commits into the
"validated" repo.  This might be best as the test process could do it
automatically, and simply delete anything that causes a test failure
or merge conflict.  Sprinters whose commits don't make it will have to
come back and fix them later.  The test bot could tag the busted
commits "FAILED".  This has the disadvantage that commits could fall
on the floor if sprinters are inattentive.  It also loses branch
structure, and if (say) the 3rd commit in a series is busted, you
want to back out 1 and 2 as well.  (It's not always necessary, but I
don't see how the 'bot can know which is which.)

And of course, step 7: when all the problems are resolved, reset both
push-target and pull-source to resolve to the validated repo.
Sprinters will have to re-clone.

Step 8: convert the old push-target repo to hex as ASCII armor, and
mail it to Barry Warsaw and Ben Finney because they might want to look
at branch history.<wink>

I suspect this is more or less what the Bazaar project's PQM does.