[Python-Dev] [Distutils] how to easily consume just the parts of eggs that are good for you

[zooko]

On Apr 8, 2008, at 9:41 PM, Phillip J. Eby wrote:
>
> I'm curious.  Have any of you actually read PEP 262 in any detail?

I read it, though not in fine detail.

I didn't write that you are planning to reinvent apt.  I wrote that  
when programmers hear about this PEP they exclaim "They are planning  
to reinvent apt!".

That is a matter of perception and marketing -- the value that I want  
to gain from Python packages is the value of a critical mass of good  
programmers using compatible tools for code re-use.  If a lot of  
programmers hate an idea, then it doesn't matter what the details are  
-- it isn't going to provide this value to me.

I think part of our disagreement is that we are talking about two  
overlapping use cases: programmer and sysadmin (and "end user" which  
is much like sysadmin).  I am not, at this time, interested in the  
sysadmin use case.  As I've mentioned, my sysadmin needs are  
currently well satisfied by apt (and sometimes by GNU stow), and my  
fellow sysadmins with whom I work are absolutely not going to relax  
their "apt-only policy" for the forseeable future, so I cannot use  
such a tool unless it is named "apt" and written by Debian/Ubuntu.

On the other hand I am very interested in the programmer use case,  
because setuptools/easy_install already works pretty well for that,  
and we are already very close to achieving a critical mass of good  
programmers.  Recently several more packages that my project [1]  
relies on have become easy_installable -- Twisted, pywin32 (thanks to  
you, PJE), and foolscap -- and several more have had bugfixes which  
make them work better with easy_install/setuptools -- Nevow and  
zope.interface -- and there are some patches in the queue to make  
another one compatible with setuptools -- pyflakes [2, 3].

So setuptools/easy_install is already (slowly) winning.  I want to  
accelerate that process by reducing the degree to which it is  
incompatible, inconvenient, or objectionable to other programmers.

PEP 262 sounds like a non-starter to me because

1.  It appears to be backwards-incompatible with setuptools/ 
easy_install/eggs, thus losing all the recently gained cooperation  
that I mentioned in the previous paragraph, and

2.  It defines a new database file, where I would prefer either:
    a.  Doing away with database files entirely and relying on the  
filesystem alone to hold that information, or
    b.  Continuing to use the current ".pth" database file format,  
possibly improved by having native support for .pth files in the  
Python import machinery.

3.  Because of #2, it triggers programmers to exclaim "They are  
planning to reinvent apt!", thus making it unlikely that the new  
proposal will recapture the cooperation that setuptools has already  
(slowly) gained.

I'm sorry, PJE -- I know it must be frustrating to you to have your  
proposal criticized on social rather than technical grounds -- but  
social benefits are what I am seeking right now.

Perhaps PEP 262 and my proposal are not actually alternatives, but  
are complementary.  I do not object to Python maintaining a database  
of installed packages for itself (although I cannot *rely* upon such  
behavior, not least because I will be maintaining backwards  
compatibility with Python 2.4 for at least the next several years,  
and with Python 2.5 for at least the next several years after that).  
What I want is for the already implemented, tested, and deployed code- 
re-use features of setuptools/easy_install to be more widely  
accepted.  This is best and most easily achieved by fixing the two  
most frequent objections to setuptools/easy_install: 1.  That you  
can't conveniently install into an arbitrary directory, and 2. that  
it subverts the meaning of your PYTHONPATH.

Regards,

Zooko

[1] http://allmydata.org/source/tahoe/trunk/docs/install.html
[2] http://divmod.org/trac/ticket/2535
[3] http://divmod.org/trac/ticket/2048