[Python-Dev] [Distutils] how to easily consume just the parts of eggs that are good for you

[Phillip J. Eby]

At 10:30 AM 4/9/2008 -0700, zooko wrote:
>PEP 262 sounds like a non-starter to me because
>
>1.  It appears to be backwards-incompatible with setuptools/ 
>easy_install/eggs, thus losing all the recently gained cooperation
>that I mentioned in the previous paragraph, and

No.  It provides a forward-compatibility path for the distutils, so 
that easy_install doesn't have to install things in .egg format in 
the future.  There's no compatibility breakage at all.


>2.  It defines a new database file,

No, it defines *files*.  One file per installed distribution, 
containing (among other things) an installation manifest.


>where I would prefer either:
>    a.  Doing away with database files entirely and relying on the
>filesystem alone to hold that information, or

...which is what PEP 262 *does*.

Unfortunately, PEP 262's title is bad for marketing, as you've 
effectively pointed out.  It would be better titled something 
"package installation manifests" or "package contents files", or 
something of that sort.

>    b.  Continuing to use the current ".pth" database file format,
>possibly improved by having native support for .pth files in the
>Python import machinery.

These mechanisms are orthogonal to this issue.


>3.  Because of #2, it triggers programmers to exclaim "They are
>planning to reinvent apt!", thus making it unlikely that the new
>proposal will recapture the cooperation that setuptools has already
>(slowly) gained.

Yeah, we need a new name.  Everybody is going off of "database of 
installed packages" and thinking "apt", because they aren't paying 
any closer attention.  However, given that we are discussing this on 
Python-Dev and distutils-sig, I do think it's reasonable to expect 
(if perhaps not reasonable to receive) that people discussing the PEP 
have *read* the freaking PEP first, prior to trashing it or offering 
alternatives.

And it's not like I'm personally offended or anything -- I didn't 
even write the PEP in question.  But what's the point of having PEPs 
if people read nothing but their titles?  We could just delete 
everything but PEP 0.  :)


>Perhaps PEP 262 and my proposal are not actually alternatives, but
>are complementary.

As I've already pointed out, your proposal does not address multiple 
installed versions of a package, and I see no sane way to modify it to do so.


>What I want is for the already implemented, tested, and deployed 
>code- re-use features of setuptools/easy_install to be more widely
>accepted.  This is best and most easily achieved by fixing the two
>most frequent objections to setuptools/easy_install: 1.  That you
>can't conveniently install into an arbitrary directory, and 2. that
>it subverts the meaning of your PYTHONPATH.

As I've already stated, the only way for these problems to be fixed 
is for easy_install to not install files in .egg form -- which also 
solves the general objection to using .eggs in the first place.

And the only way to do that, is to have a way to keep track of what 
files are installed.  Rather than have easy_install come up with its 
own way of doing that, I would prefer to share a standard with the 
distutils.  Hence, the PEP discussion.

For earlier versions of Python, it will still be possible to install 
and uninstall with setuptools using this approach.  You just won't be 
able to uninstall pure distutils-based packages, unless you installed 
them using easy_install.

Meanwhile, it has occurred to me that the easiest way of handling 
compatibility is not to require that other packaging tools mark their 
files for non-removability, but simply not allow easy_install to 
remove or overwrite anything that *isn't* claimed by a manifest.  In 
that way, easy_install would be immediately usable in the new mode, 
without any updates to Python or to system packaging tools.