[Python-Dev] PEP 376

[P.J. Eby]

At 02:54 PM 7/3/2009 +0100, Paul Moore wrote:
>Eggs are fundamentally a PEP 302 zip file format. There are some 
>extra bits of metadata for setuptools/easy_install in there (as I 
>understand things) but essentially they are zip files. When you say 
>"decoupling the egg format", I assume you mean "decoupling the egg 
>metadata" - which is fine, but to properly decouple, you need API 
>level access to the metadata. PEP 376 offers read-only access, but 
>as you rightly point out, it is only for filesystem data (and some 
>form of zip file, which appears to be limited in some way, as it 
>isn't PEP 302 based, and the actual format isn't defined anywhere). 
>The basic point here is that PEP 376 needs to define precisely how 
>pkgutil.get_distributions() scans sys.path looking for ".egg-info 
>directories". What does it do for sys.path entries that don't 
>correspond to filesystem directories? (Note - these may or may not 
>be zip files. Even if they are zip files, an earlier entry on 
>sys.path_hooks could have taken precedence. At the very least, you 
>should only process path entries as zip files if their importer - in 
>sys.path_importer_cache or via an explicit path hook scan - is a 
>zipimporter object.). To be honest, this is a major can of worms. 
>But if PEP 376 is not going to support PEP 302, then it must state 
>that fact explicitly, to avoid giving people false expectations - 
>particularly with Brett's importlib in Python 3.1, which will make 
>it far easier for people to experiment with new packaging formats 
>such as the ones Lukasz mentions above. And it MUST fail gracefully 
>in the face of unsupported importer types.

Well, we could always resurrect PEP 365, since pkg_resources already 
has documented extensible support for arbitrary importers.  That 
solves backward *and* forward compatibility.  Then PEP 376's 
uninstall facilities could be implemented using pkg_resources' 
existing metadata query features.

The primary downside to that, of course, is that it brings in the 
matter of version specifications and dependencies...  which appear to 
be a contentious topic.  (Note that Tarek is proposing to drop the 
PEP 386 proposal to standardize a much more restrictive scheme than 
seutptools' version parser, precisely because of the controversy.)