[Distutils] [Fwd: Re: Annoucing distribute project]

[Guido van Rossum]

On Thu, Sep 25, 2008 at 1:59 PM, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
> On Thu, Sep 25, 2008 at 7:36 PM, Guido van Rossum <guido at python.org> wrote:
>>
>> I don't think anyone loves distutils, or even likes it. Unfortunately
>> replacing it will be very tough, because distutils has arcane but
>> important knowledge built into it about many platforms, and a new
>> from-scratch system is unlikely to reach that level of coverage for
>> years. There's also the issue that it's not enough to roll out the
>> replacement with a new version of Python -- it would leave everyone
>> using the old distutils in the lurch. It's a prime example of the
>> network effect: until everyone owns one, nobody will want to own one.
>> The best strategy I can think of would be to gradually rewrite the
>> existing distutils, remaining compatible but also offering new APIs,
>> and 5 years hence maybe the installed base will be close enough to
>> 100% that developers writing setup scripts will start using the new
>> features. That's how AJAX conquered the world.
>
> It looks like you are describing setuptools in some ways: it was written
> on the top of distutils and provided enhancements and new features.

The mistake made was that setuptools didn't make it into the core
distro. That in turn was because of concerns over its maintenance once
incorporated into the core. (There's no need to go into more details.)

> But in the meantime distutils itself did not evolve.

Because nobody offered to evolve it gradually (setuptools was an
attempt at a quantum leap, which I was trying to explain is the wrong
approach).

> As far as I know, within the last year, one of the only new feature in
> distutils was the one
> I have proposed (multiple servers support in .pypirc).  But it was really
> hard to get
> my change make it in because most  Python core developers have a lack of
> interest for
> distutils. It took me 8 months iirc.

So you need to work yourself up to becoming a core developer. That's
not impossible -- it just means contributing to other areas besides
distutils.

> Furthermore there are many things to change in this module to make it use
> modern
> Python before starting to add new features. (like logging usage but this is
> just an
> example)

The best thing I can recommend is to start by contributing relatively
small cleanups, that are easy to review and apply.

> My point is that I am afraid not much will happen if something doesn't start
> outside Python
> core.

My worry is that if something develops outside the core without
careful thought about migration and compatibility it will be
impossible to adapt into the core.

A metaphor:

You're not building a brand new bridge where there was no bridge
before. You're trying to upgrade an existing bridge that people use
for their daily commute. You can't even close a single lane during
rush hour -- you have to plan carefully how the new bridge can grow
organically right next to the old one and how traffic can be kept
flowing continuously. Yes, it is much more expensive than just
building a new bridge somewhere else. But it is the only way to do it.
So you have to figure out how to do it.

> Right now there's a momentum in the community, including framework gurus,
> that are
> willing to work on a new distutils package. They are not core developers but
> they are really
> good in distribution matters. Even Phillip Eby said that starting a new
> distutils could be a good
> pick in this thread earlier.

I wasn't there. I'd like to refer to a post by Joel Spolsky about the
problem with total rewrites:
http://www.joelonsoftware.com/articles/fog0000000069.html

> Maybe a "distutils 2" project could start outside Python, using distutils
> and setuptools code as
> legacy infrastructure, and taking back pieces of it,
>
> Then it could be re-integrated in Python as a replacement for distutils when
> it is mature ?

Only if much effort and study went into the planning of this re-integration.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)