[Python-Dev] Should there be a way or API for retrieving from a code object a loader method and package file where the code comes from?

[Paul Moore]

2008/12/23 Nick Coghlan <ncoghlan at gmail.com>:
> Finding a loader given only a pseudo-filename and no module is actually
> possible in the specific case of zipimport, but is still pretty obscure
> at this point in time:
>
> 1. Scan sys.path looking for an entry that matches the start of the
> pseudo-filename (remembering to use os.path.normpath).
>
> 2. Once such a path entry has been found, use PEP 302 to find the
> associated importer object (the undocumented pkgutil.get_importer
> function does exactly that - although, as with any undocumented feature,
> the promises of API compatibility across major version changes aren't as
> strong as they would be for an officially documented and supported
> interface).
>
> 3. Hope that the importer is one like zipimport that allows get_data()
> to be invoked directly on the importer object, rather than only
> providing it on a separate loader object after the module has been
> loaded. If it needs a real loader instead of just the importer, then
> you're back to the original problem of needing a module or package name
> (or globals dictionary) in addition to the pseudo filename.

There were lots of proposals tossed around on python-dev at the time
PEP 302 was being developed, which might have made all this easier.
Most, if not all, were killed by backward compatibility requirements.

I have some hopes that when Brett completes his "import in Python"
work, that will add sufficient flexibility to allow people to
experiment with all of this machinery, and ultimately maybe move
forward with a more modular import mechanism. But the timescales for
Brett's changes won't be until at least Python 3.1, and it'll be a
release or two after that before any significant change can be eased
in in a compatible manner. That's going to take a lot of energy on
someone's part.

Paul.

PS One of these days, I'm going to write an insanely useful importer
which takes the least-convenient option wherever PEP 302 allows
flexibility. It'll be adopted by everyone because it's so great, and
all the software that currently makes unwarranted assumptions about
importers will break and get fixed to support it because otherwise its
users will rebel, and we'll live in a paradise where everything
follows the specs to the letter. Oh, yes, and I'm going to win the
lottery every week for the next month :-)

PPS Seriously, setuptools and the adoptions of eggs has pushed a lot
of code to be much more careful about unwarranted assumptions that
code lives in the filesystem. That's an incredibly good thing, and
very hard to do right (witness the setuptools "zip_safe" parameter
which acts as a get-out clause). Much kudos to setuptools for getting
as far as it has.