[Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration)

[Stephen J. Turnbull]

anatoly techtonik writes:

 > After reading PEP 384 and PEP 385 (finally) I got a strong impression
 > that they are not ready for the change (read below the line for
 > details), because they do not propose any workflow.

This was deliberate.  There are a lot of different workflows, and we
are not in a position to tell people to change them.

 > So, instead of rushing with migration

We are hardly rushing.  You're just late to the party.  Catch your
breathe, get a beer, have a good time.  No?

 > I'd like to propose incremental change rather than revolutionary
 > that will allow people to taste what it like to work with Mercurial
 > and update all relevant papers on the website.

Been there, done that, if that's the way it's gonna be, I won't touch
it.  It's much too painful for those who are doing the work, compared
to the small benefits received by the community as a whole.

 > I am willing to collaborate for getting this system in place before
 > EuroPython even if I won't find a chance to get there.

Are you also willing to maintain the system indefinitely?  I haven't
yet seen such a system that really works without regular attention
from a human.

 > However, PEP 385 "The Future of Subversion" [1] does not propose any
 > way to keep Subversion repository in sync. "it will probably live on
 > for a bit" phrase looks like Subversion will be immediately dead after
 > switching.

Yes, that is the intent.  For old versions, the Subversion repo will
be available read-only.

For casual contributors, there's a one-time set up cost of pulling a
local mirror from the public repository.

For those familiar with Mercurial, there is nothing but joy to be
found in the transition.

For folks like you, heavily invested in Subversion-based private
automation, yes, there will be some pain.

 > The second concern is workflow. Even after getting my hands dirty I
 > still can't find the best solution for working with Mercurial copy of
 > Python. In Subversion I do shallow checkouts of interesting features
 > when I run into one bug or another. Not all patches are ready, so they
 > stay in those checkouts as uncommitted diffs.

For this kind of workflow, Mercurial queues (mq) is probably the best
solution.  Unfortunately, it will require you to write some additional
scripts to get the same level of convenience you have now.

 > To be prepared for conflicts I review code with `svn log -vr
 > base:head` before updating.  But with Mercurial I see two major
 > problems with my workflow (which I am unlikely to change for the
 > next few weeks due to heavy automation):
 > 1. No shallow checkouts - that means that I have to copy 200Mb+ for
 > every patch I have

No, you don't.  You make links to 200MB+ (unless you're on Windows,
where I don't know how this works).  This is much cheaper than
copying, though not as cheap as in git.  I don't hesitate to make
branches in Mercurial.

Alternatively, for localized bugfixes, doc typos, and the like, one
"little-fixes" branch with an mq patch or two for each is a reasonable
way to work.

 > 2. No `hg log` command to see changes in my branch from remote
 > repository before fetching them
 > (I used to invoke `hg incoming`, but ended up with fetching everything
 > with `hg pull` and then using `hg log -vr .:tip`, because `hg inc`
 > didn't show changes that were already pulled, but not updated in wc,
 > and `hg log` could not show incoming)

 > 3. There is no clear branch separation in my working copy. I don't
 > know how to diff files in different branches without copying 200Mb+
 > clone. I don't know how to diff my files with remote repository
 > without pulling it.

If you're at all serious about working with Mercurial, you don't do
any operations except pull (and push, if you're a committer) against
the remote.  You keep a local pristine mirror of the remote
repository, which you update frequently, and all of your work revolves
around that rather than around contact with upstream.

This does mean that when upstream uses a repo-per-branch model you
have to keep multiple mirrors.  On the other hand, you can work
directly in branches that you work on only infrequently, and also use
Mercurial queues to avoid making branches if you don't want to.

 > 4. Even if I make patch in my Mercurial clone - you still can't pull
 > it and I have to attach it to tracker. No gain.

No loss, either, since that's the current workflow, and maintaining as
much of the current workflow (possibly with "new spellings" of the
commands implementing it) was an explicit goal for PEP 374.  I assume
that carries over to PEP 385.

If you and some others working on a common project want to work with
branches rather than patches (email or tracker), then if there's a
committer in the group, you can set up a branch on code.python.org.
If not, you can use bitbucket or something.  Yes, it's a bit of extra
effort for you.

 > So from these PEPs I can't really see how Mercurial solves problems
 > with development. I can feel that it doesn't solve my problems, but I
 > see no way where I can submit my user story.

It doesn't solve problems with development, and that is not the intent
of the PEPs AIUI.  IMO, the theme of these PEPs is to maintain current
practices and gives a platform for improved practices in the future.

 > I would put accent on keeping mirror of Subversion as easy way to
 > contribute for those who are not yet ready for DVCS.

"Accent" won't cut it.  Effort is needed.  That effort won't be mine,
I don't know how anybody else likely to be involved feels about it.

And in fact, Mercurial makes it quite a bit *easier* for *casual*
contributors to work on the code base.  *All* of my work based on
projects using Subversion (or CVS) is done from colocated DVCS
branches.  The only difference from my basic DVCS workflow is
(1) the synch to upstream command is "svn up" rather than "hg pull
-u", and (2) those changes must be committed to the DVCS (automated by
script).

It also is clearly going to make more effective workflows for many of
the core developers.  AFAIK, assuming the issues that have been raised
in PEP 385 and the tracker are resolved, other core developers agree
that the transition will have an acceptably low impact on them.

Admittedly, it seems this transition is going to affect you
adversely.  That's unfortunate, but AFAIK you are in a small minority.

Note that this is my current understanding of the situation.  As the
transition proceeds (and we are still very much in a planning phase,
where anything could be changed), it may become clear that active
developers in your situation are more numerous than I (we?) think.
Also, any suggestions you make, including maintaining the Svn
repository, will be taken seriously ... but many of them, specifically
the decision to freeze the Svn repo once we cut over, have already
been thought through carefully.  Doesn't mean that they can't be
changed, but you do have to accept that you're going to need to
provide evidence that changing the decisions will benefit the
community as a whole.