[Python-Dev] PEP 376

[Paul Moore]

2009/7/3 Nick Coghlan <ncoghlan at gmail.com>:
> I agree that even if the reference implementation of PEP 376 only
> handles normal directories and zipfiles, the PEP itself needs to explain
> how the developer of a custom PEP 302 importer or loader can hook into
> the mechanism in order to provide metadata that distutils can
> understand. Or, as you suggest, state explicitly that the proposal at
> this stage is specifically limited to filesystem and zipfile packages
> and defer extension to arbitrary PEP 302 importers and loaders to a
> later PEP.
>
> To be honest, I'd personally be OK with the latter strategy - while
> other PEP 302 loaders and importers do exist (as Lukasz'z post shows),
> it would be unfortunate to unduly hold up improved installation metadata
> for the vast majority of typical use cases while we try to figure out
> ways to support the more esoteric use cases. Supporting both would
> obviously be better, but given the choice between the status quo and
> partial support, I believe this is a case where the partial support
> would still be worthwhile.

You put it more clearly than I did. That's basically what I think,
with the one proviso that we should make sure that PEP 376 doesn't
specify something that out and out breaks the more esoteric PEP 302
cases.

When Just and I were developing PEP 302, we found that the best way to
do that was to leave anything that didn't need to be specified, as
unspecified (hence the fact that there's so little defined in PEP
302!). It's easier to add things later than to remove or change them.
That's why I was recommending to Tarek that he take out of the PEP any
details about classes or APIs that couldn't be directly obtained from
the core API. The same applies here (just talking in terms of a
duck-typed notional Distribution class, allows people to implement
their own URLDistribution or SqliteDistribution, or whatever, and not
have to emulate any more of a filesystem than necessary).

> I will note (and I believe this is also the main point that Lukasz was
> making) that having the distribution metadata outside the distribution
> as currently proposed in PEP 376 is going to make any eventual PEP 302
> integration much harder - 302 importers only provide a mechanism for
> accessing files inside the distribution, not "adjacent" to them, so the
> mechanism in the PEP doesn't generalise properly.
>
> I suspect this limitation of the PEP 302 APIs is the origin of the
> setuptools format that embeds the metadata inside the distribution - it
> lets you get at the metadata without having to assume that it exists
> directly on the filesystem anywhere.

Again, agreed. But remember that PEP 302 is driven by looking up a
module or package name. PEP 376 is looking up a *distribution* name.
The docutils example in the PEP shows this:

- docutils/
- roman.py
- docutils-0.5-py2.6.egg-info/

There are 3 things here: a package (docutils), a module (roman) and a
distribution (also named docutils, but could be called George for all
it matters). So none of the PEP 302 lookup mechanisms (which work on
package/module name) apply.

The need for a separate concept (a "distribution") is unfortunate, as
it adds complexity, but there are enough real life cases that make it
clear that it's necessary.

Hmm, I suspect that the implication here is that PEP 302 could do with
an overhaul, to extend it to encompass the concept of a distribution.
I'd be willing to have a look at that.

Paul.