[Distutils] .egg in Debian summary?

[Phillip J. Eby]

At 04:39 PM 11/23/2005 +0100, Vincenzo Di Massa wrote:
>I think setuptools could really help Debian developer as well as Fedora,
>Mandriva, Gentoo... by:
>1) providing functionality to install files outside of egg dir/zip (for
>example one could install code to site-packages and data-files to /usr/share/
>or somewhere else)

There will be some steps towards this in the 0.7 series, although initially 
that's going to be by providing user-executed post-installation tools, 
which would also be executable by packager-defined postinstallation 
steps.  Since easy_install is not a package manager, however, I would 
prefer users be able to trust that running easy_install is only going to 
make a Python package and some scripts available, and not do anything to 
files elsewhere on their system.  This position may evolve over time, since 
there are a lot of other features that (in my mind at least) need to come 
first before I can think deeply about the issue.  (Related issues: header 
file installation, documentation, etc.)


>2) providing a "fakeroot" option to fool setup.py (for example I want
>setuptools to install the files in debian/tmp/usr/lib/pythonX.Y/site-packages
>and scripts in debian/tmp/usr/bin and everithing must work when it copied
>to /usr/{bin,lib}.

There is in fact such an option now; as far as I know the --root option to 
"setup.py install" should do it, and I am pretty sure it's used by bdist_rpm.


>2) providing a paseable output with setup process data like: dependencies
>(meet and unmett), files installed, author, license, and so on

Perhaps you could provide a sample of what you have in mind?  A patch would 
be even better, although I would be using it mainly to understand what 
you'd like to have and not necessarily as a basis for implementing it.


>All this will let people just "convert" the output of setuptools into an rpm
>specfile, debian dir, gentoo ebuild and so on.
>All that is needed is just a common metadata format that can be parsed by
>tools like easy-deb, easy-rmp easy-ebuild easy-(???).

Note that easy_install and install have --record options that write the 
names of all installed files or directories to a file for you.  The 
remaining metadata you're asking about can already be queried via the 
pkg_resources API, since it's part of the resulting 
eggs.  'someEgg.get_metadata("PKG-INFO")' will get you the PKG-INFO file 
with all the standard distutils metadata, for example.  The 'requires()' 
method gives information about dependencies.


>This way setuptools provides the information to distribution maintainers in a
>well documented, api independent way.

Well, if you want API independence, you can directly look at the .egg-info 
or EGG-INFO directory contents, as the data is in plain-text files like 
requires.txt and PKG-INFO.  PKG-INFO is a PEP-defined format, and the 
requires.txt format has been stable for maybe 7 months now.  I don't 
anticipate any changes to it.


>  Distribution maintainers can use this
>info to reorganize, repackage, invent new package formats, but without
>involving setuptools and python developers.

Fair enough, although you should keep in mind that there are only three 
valid installation layouts for egg-specific projects: .egg zipfile, .egg 
directory, and "old school" installation with an .egg-info directory 
installed alongside.  Anything else, and it would involve changing the egg 
runtime to work with it.


>About the pythonpath and pth files handlind:
>if a project A really wants to use project B eggs it can easily require them
>without the need for a pth file.
>If project C wants to use project B, but project C is not setuptools aware 
>one
>could patch project C (just adding require() to it) instead of paching
>project B and project A removing egg support from them.

Note that if C is an application rather than a library, one can add the 
requirements to setup.py and use easy_install to install it, which will 
create script wrappers that perform the necessary require() 
operations.  So, you don't have to patch C very deeply in this case.  In 
addition, the scripts will require() C itself, so you don't need to install 
C with a .pth unless it is a library that is needed by something else that 
is not packaged using this system.

Basically, the whole thing is designed so that as long as a project is 
setuptools-based, it doesn't need explicit require() calls, as these are 
handled by setuptools.  Only if you want a project to be *always* on 
sys.path do you need a .pth file.


>Onother possible solution would be to have:
>python-A
>python-pypi-A
>python-B
>python-pypi-B
>python-C
>
>Where
>python-B) depends on python-pypi-A and just provides the pth file (handled
>like easy-deb does)
>python-pypi-B) contains the egg
>
>So project C will depend on python-B, which will install the pth. But if
>project B is required just by project A which is setuptools aware the
>dependency will be on python-pypi-B: the pth file will not be installed.

So, you're saying that python-foo would install 'foo' such that it is 
always on sys.path, but python-pypi-foo simply installs 'foo' in such a way 
that is discoverable by the egg runtime.  That's an interesting idea, I'll 
have to think a while about the consequences of that before I comment on it 
further.