[SciPy-dev] [matplotlib-devel] Announcing toydist, improving distribution and packaging situation

[David Cournapeau]

On Tue, Dec 29, 2009 at 3:03 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> David Cournapeau wrote:
>
>> On Mon, Dec 28, 2009 at 11:47 PM, Stefan Schwarzburg
>> <stefan.schwarzburg at googlemail.com> wrote:
>>> Hi,
>>> I would like to add a comment from the user perspective:
>>>
>>> - the main reason why I'm not satisfied with pypi/distutils/etc. and why
>>> I will not be satisfied with toydist (with the features you listed), is
>>> that they break my installation (debian/ubuntu).
>>
>> Toydist (or distutils) does not break anything as is. It would be like
>> saying make breaks debian - it does not make much sense. As stated,
>> one of the goal of giving up distutils is to make packaging by os
>> vendors easier. In particular, by allowing to follow the FHS, and
>> making things more consistent. It should be possible to automatically
>> convert most packages to .deb (or .rpm) relatively easily. When you
>> look at the numpy .deb package, most of the issues are distutils
>> issues, and almost everything else can be done automatically.
>>
>> Note that even ignoring the windows problem, there are systems to do
>> the kind of things I am talking about for linux-only systems (the
>> opensuse build service), because distributions are not always really
>> good at tracking fast changing softwares. IOW, traditional linux
>> packaging has some issues as well. And anyway, nothing prevents debian
>> or other OS vendors to package things as they want (as they do for R
>> packages).
>>
>> David
>
> I think the breakage that is referred to I can describe on my favorite
> system, fedora.
>
> I can install the fedora numpy rpm using yum.  I could also use
> easy_install.  Unfortunately:
> 1) Each one knows nothing about the other
> 2) They may install things into conflicting paths.  In particular, on fedora
> arch-dependent things go in /usr/lib64/python<version>/site-packages while
> arch-independent goes into /usr/lib/python<version>...  If you mix yum with
> easy_install (or setuptools), you many times wind up with 2 versions and a
> lot of confusion.
>
> This is NOT unusual.  Let's say I have numpy-1.3.0 installed from rpms.  I
> see the announcement of numpy-1.4.0, and decide I want it, before the rpm is
> available, so I use easy_install.  Now numpy-1.4.0 shows up as a standard
> rpm, and a subsequent update (which could be automatic!) could produce a
> broken system.

Several points:
 - First, this is caused by distutils misfeature of defaulting to
/usr. This is a mistake. It should default to /usr/local, as does
every other install method from sources.
 - A lot of instructions start by sudo easy_install... This is a very
bad advice, especially given the previous issue.

> I don't really know what could be done about it.  Perhaps a design that
> attempts to use native backends for installation where available?

The idea would be that for a few major distributions at least, you
would have .rpm available on the repository. If you install from
sources, there would be a few mechanisms to avoid your exact issue
(like maybe defaulting to --user kind of installs). Of course, it can
only be dealt up to a point.

David