[Python-Dev] Source control tools

Mon Jun 12 23:39:45 CEST 2006

On 6/12/06, Thomas Wouters <thomas at python.org> wrote:
>
>
> On 6/12/06, Guido van Rossum <guido at python.org> wrote:
>
> > Perhaps issues like these should motivate us to consider a different
> > source control tool. There's a new crop of tools out that could solve
> > this by having multiple repositories that can be sync'ed with each
> > other. This sounds like an important move towards world peace!
>
>
> It would be an important move towards world peace, if it didn't inspire
> whole new SCM-holy-wars :-)  I have a fair bit of experience with different
> SCM (VC, source control tool, however you want to call them) so I'll take
> this opportunity to toss up some observations. Not that switching to another
> SCM will really solve the issues you are referring to, but I happen to think
> switching to another SCM is  a great idea :) Although I don't see an obvious
> candidate at the moment... I'll explain why.
>
> First of all, changing SCM means changing how everyone works. It's nothing
> like the CVS->Subversion switch, which changed very little in workflow. All
> the cool SCMs use 'real branches', and to get full benefit you have to
> switch your development to a 'branch oriented model', if you'll pardon the
> buzzwordyness. At XS4ALL we've used BitKeeper for a few years now, and while
> it really took a while for some of the developers to catch on, the branch
> thing makes parallel development *much* easier. If you haven't experienced
> it yourself, it's hard to see why it matters so much (except maybe in cases
> with extreme parallel development, like the Linux kernel), but it really
> does make life a lot easier in the long run, for programmers and release
> managers.
>
> Secondly, the way most of the 'less-linear' SCMs work is that everyone has
> their own repository. That is, in fact, what makes it so useful -- no need
> for central repository access, no need for a network connection for full
> operability, no need for write access to get your changes checked in
> (locally), and easy interchanging between repositories. With a large
> (history-rich) project like Python, though, those repositories can get a tad
> large. Most of the SCMs have ways to work around it (not downloading the
> full history, side-storing the full history in a separate branch, etc) but
> it's still an extra inconvenience. Now, me, I don't mind downloading a
> couple hundred megabytes to get a 25Mb sourcetree with full history, but I
> have a 1Gbit link to the python.org servers :) On the other hand, with
> most of the SCMs, you only download that full history once, and then locally
> branch off of that.
>
> The real reason I think we should consider other SCMs is because I fear
> what the history will look like when 3.0 is done. In Subversion, merges of
> branches don't preserve history -- you have to do it yourself. Given the way
> Subversion works, I don't think that'll really change; it's just not what
> Subversion is meant to do (and that's fine.) It does mean, however, that
> when we switch the trunk to 3.0, we have to decide between the history of
> the trunk or the history of the p3yk branch. We either merge the p3yk branch
> into the trunk, making a single huge checkin message explaining all the
> changes (or not), or we swap the trunk and the p3yk branch. The former means
> 'svn blamelog', for instance, will show a single revision and a single
> author for *all* p3yk changes. The latter means 'svn blamelog' will group
> the trunk changes into the merge commits you can already see on the
> python-3000-checkins list: a block of merges at a time, based on whenever I
> (or someone else) has the time to merge the trunk in. So, in that case, 'svn
> blamelog' will show *me* as author of all 2.5-to-2.7 changes, at a time
> the original change didn't go in, with log messages that are largely
> irrelevant ;-) And the mess gets bigger if part of p3yk or trunk's
> development is done in other branches -- svnmerge log messages hidden in
> svnmerge log messages. ugh.
>
> Before XS4ALL switched to BitKeeper, I spent quite a while examining
> different SCMs, but at the time, they just weren't at the stage of
> development you'd trust your company development to (not if you can afford
> BitKeeper anyway ;)  After (re-)experiencing the pain that is
> Subversion/CVS-style branching with the p3yk branch and the manual trunk
> merges, I went ahead and checked out the current state of the alternatives.
> There are quite a few, now (Monotone, Darcs, Git/Cogito, Mercurial,
> Arch/tla/Bazaar, Bazaar-NG, Arx, CodeVille, SVK) and I haven't had time to
> give them all the in-depth examination they are worthy of, but so far it
> looks like only a few of them currently scale well enough to handle a large
> (history-rich) project like Python. Not that it's fair to expect them to
> scale well, mind you, given that most of them are only a few years old and
> most don't claim to be " 1.0".
>
> Using 'tailor' ( http://www.darcs.net/DarcsWiki/Tailor ) I imported the
> Python sourcetree with full history into Darcs and Git. I also did a partial
> import into Monotone (history going back a few months) -- the Monotone
> import was a lot slower than the others, and I didn't feel like waiting a
> week. I then made a branch of each and imported the p3yk branch into them
> (using some hand-crafting of tailor's internal data, as it doesn't seem to
> support branch-imports at the moment.) Darcs was being troublesome at that
> point, but I haven't spent the time to figure out if it was something I or
> tailor did wrong in the import. As I said, Monotone was rather slow, which
> is not surprising considering it does a lot of signing of digital
> certificates. I personally like Monotone a lot, because its central
> branch-database is the 'next step up' from what most SCMs do and because I
> really like the cryptographic aspect, but it's probably too complex for
> Python. Git, the 'low level SCM' developed for the Linux kernel, is
> incredibly fast in normal operation and does branches quite well. I suspect
> much of its speed is lost on non-Linux platforms, though, and I don't know
> how well it works on Windows ;)
>
> I did partial imports into Mercurial and Bazaar-NG, but I got interrupted
> and couldn't draw any conclusions -- although from looking at the
> implementation, I don't think they'd scale very well at the moment (but that
> could probably be fixed.) I should also look at SVK (a BitKeeper-style SCM
> ontop of Subversion, really), but tailor doesn't support it (yet) and my
> last look was just too depressing to cope with manually importing Python.
>
> In short[*], I don't see an immediate candidate for an alternate SCM for
> Python (although Git is sexy), but there's lots of long-term possibilities.
> I intend to keep my Git and Darcs repositories up to date (it's little
> effort to make tailor update them), tailorize Mercurial, Bazaar-NG, (full)
> Monotone and probably others, tailorize some branches as well, and publish
> them somewhere, hopefully with instructions, observations and honest
> opinions (I just need to find the right place to host it, as e.g. Monotone
> really wants a separate daemon process to run.)  I also intend to do my own
> p3yk development in one of those SCMs; I can just export patches and apply
> them to SVN when they're ready ;P I would like to hear if others have any
> interest at all in this, though, if anything to keep me motivated during the
> tediously long tailorizing runs :)
>

I am interested, especially in Bazaar-NG.  Martin Poole's (author of
Bazaar-NG) talk at PyCon was good.  Plus Canonical might be willing to help
us out if we ever went with Bazaar (I believe they stated this once back
when the cvs->svn move was happening).

-Brett

Oh, and in case it matters to people: tailor, Mercurial and Bazaar-NG are
> written in Python.
>
> Blurt-blurt'ly y'rs,
>
> [*] Short? This whole mail was short! I can talk for hours about the
> benefit of proper branches and what kind of stuff is easier, better and more
> efficient with them. I can draw huge ASCII diagrams explaining the
> fundamental difference between CVS/SVN,
> BitKeeper/Arch/Darcs/Bazaar-NG/Mercurial and Monotone (yes, that's three
> groups), and I have powerpoint(!) sheets I used to give a presentation about
> how and why BitKeeper works at work. It's probably a bit off-topic here,
> though, so don't tempt me ;P
> --
> Thomas Wouters <thomas at python.org>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me
> spread!
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060612/b1f6fc97/attachment.html 