[Python-Dev] PEP: Migrating the Python CVS to Subversion

["Martin v. Löwis"]

I'd like to see the Python source be stored in Subversion instead
of CVS, and on python.org instead of sf.net. To facilitate discussion,
I have drafted a PEP describing the rationale for doing so, and
the technical procedure to be performed.

This is for discussion on python-dev and eventual BDFL pronouncement;
if you see a reason not to make such a change, or if you would prefer
a different procedure, please speak up. Encouragement and support is
welcome, as well :-)

Regards,
Martin

PEP: XXX
Title: Migrating the Python CVS to Subversion
Version: $Revision $
Last-Modified: $Date$
Author: Martin v. Löwis <martin at v.loewis.de>
Discussions-To: <python-dev at python.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 14-Jul-2004
Post-History: 14-Jul-2004


Abstract
========

The Python source code is currently managed in a CVS repository on
sourceforge.net. This PEP proposes to move it to a subversion repository
on svn.python.org.

Rationale
=========

This change has two aspects: moving from CVS to subversion, and
moving from SourceForge to python.org. For each, a rationale will
be given.

Moving to Subversion
--------------------
CVS has a number of limitations that have been elimintation by
Subversion. For the development of Python, the most notable improvements
are:
- ability to rename files and directories, and to remove directories,
  while keeping the history of these files.
- support for change sets (sets of correlated changes to multiple
  files) through global revision numbers.
- support for offline diffs, which is useful when creating patches.

Moving to python.org
--------------------
SourceForge has kindly provided an important infrastructure for the
past years. Unfortunately, the attention that SF received has also
caused repeated overload situations in the past, to which the SF
operators could not always respond in a timely manner. In particular,
for CVS, they had to reduce the load on the primary CVS server by
introducing a second, read-only CVS server for anonymous access.
This server is regularly synchronized, but behind the the read-write
CVS repository between synchronizations. As a result, users without
commit access can see recent changes to the repository only with
a delay.

On python.org, it would be possible to make the repository accessible
for anonymous access.

Migration Procedure
===================

To move the Python CVS repository, the following steps need to be
executed. The steps are elaborated in more detail in the next
sections.

1. Assign passwords for all current committers for use on svn.python.org.
   User names on SF and svn.python.org should be identical, unless some
   committer requests a different user name.
2. At the beginning of the migration, announce that the repository on
   SourceForge closed.
3. 24 hours after the last commit, download the CVS repository.
4. Convert the CVS repository into two subversion repositories,
   one for distutils and one for Python.
5. Publish the repositories for write access for all committers,
   and anonymous read access.
6. Disable CVS access on SF.

Assign Passwords
================

Currently, access to Subversion on svn.python.org uses WebDAV over
https, using basic authentication. For this to work, authorized
users need to provide a password. This mechanism should be used,
atleast initially, for the Python CVS as well, since various committers
also have a username/password pair for the www SVN repository already.

Alternatives to password-based access include:
- Subversion over SSH, using SSH key pairs. This would require
  to give committers accounts on the machine, which currently is
  ruled out by the administration policy of svn.python.org.
- Subversion over WebDAV, using SSL client certificates. This
  would work, but would require to administrate a certificate
  authority.

Downloading the CVS Repository
==============================

The CVS repository can be downloaded from

http://cvs.sourceforge.net/cvstarballs/python-cvsroot.tar.bz2

Since this tarball is generated only once a day, some time after
the repository freeze must pass before the tarball can be picked
up. It should be verified that the last commit, as recorded on
the python-commits mailing list, is indeed included in the tarball.

Converting the CVS Repository
=============================

The Python CVS repository contains two modules: distutils and
python. Keeping them together will produce quite long repository
URLs, so it is more convenient if the Python CVS and the distutils
CVS are converted into two separate repositories.

As the repository format, fsfs should be used (requires Subversion 1.1).
fsfs has the advantage of being more backup-friendly, as it allows to
backup a repository incrementally, without requiring to run any dump
commands.

The conversion should be done using cvs2svn utility, available e.g.
in the cvs2svn Debian package. The command for converting the Python
repository is

cvs2svn -q --encoding=latin1 --force-branch=cnri-16-start
--force-branch=descr-branch --force-branch=release152p1-patches
--force-tag=r16b1 --fs-type=fsfs -s py.svn.new python/python

The command to convert the distutils repository is

cvs2svn -q --encoding=latin1 --fs-type=fsfs -s dist.svn.new python/distutils

Sample results of this conversion are available at

http://www.dcl.hpi.uni-potsdam.de/python/
http://www.dcl.hpi.uni-potsdam.de/distutils/

Publish the Repositories
========================

The repositories should be published at https://svn.python.org/python
and https://svn.python.org/distutils. Read-write should be granted
through basic authentication to all current SF committers; read-only
access should be granted anonymously. As an option, websvn (available
e.g. from the Debian websvn package) could be provided.

The current SF project admins should get write access to the password
file, in order to create or delete new users.

Disable CVS
===========

It appears that CVS cannot be disabled entirely. Only the user interface
can be removed from the project page; the repository itself remains
available. If desired, write access to the python and distutils modules
can be disabled through a commitinfo entry.