[Python-Dev] Alternative Approach to Relative Imports

[M.-A. Lemburg]

Gordon McMillan wrote:
> 
> M.-A. Lemburg wrote:
> [msg 1]
> > Currently, the imputil apporach uses a simple chaining
> > technique. Unfortunately, it doesn't allow inspecting the chain
> > for already loaded hooks, so the same type of hook could be
> > loaded more than once.
> 
> I was hoping Greg would jump in, but since he hasn't -
> 
> You're associating the hook with the strategy. That's the old
> style. The imputil style is to associate the hook with the
> actual stuff being managed. The strategy is a property of the
> hook.

I know, but there still is no way to query what kind of hooks
are already loaded and what is worse, you cannot unload or reorder
them. I'd suggest using a list of hooks which are then
traversed in the order they appear in the list, e.g.

__importers__ = [DirectoryImporter('/usr/local/lib/python1.5'),
                 ArchiveImporter('/usr/local/lib/app.pyz'),
                 WebImporter('http://www.python.org/pylib/'),
                 PathImporter(('~/bin','~/lib'))]

This also has the advantage of being able to easily query
the importers during debugging and eliminates the need to
have a predefined attribute naming scheme (such as the one imputil
uses).

> > Also, there are at least two types of hooks:
> >
> > 1. hooks that redirect the import to some other data source
> >
> > 2. hooks that modify the way modules are searched
> >
> > Since the first variant may well also be suited to used by the
> > second, the simple chaining method probably won't be powerful
> > enough to handle it.
> 
> The top level question is "is it mine to import?". Greg provides
> a framework that makes it easy to use alternate data sources,
> and alternate ways of finding things but that's not really the
> key thing. You're a "good" importer if you can (when
> appropriate) way "no it's not mine" efficiently.

It does a good job at this, but doesn't really separate
lookup and loading of code too well. Everything is packaged
into one single method (.get_code()) which is not always
flexible enough, e.g. it wasn't possible to implement the
walk-up-the-dotted-name scheme using modifications to
.get_code() alone.

Also, I can see many uses where you combine a lookup
hook (e.g. for loading modules across the web) with
a filtering hook (e.g. one which checks a module signature).

This should go into the framework as well, IMHO...
e.g. by having two methods .find_code() and .make_module()
(like the builtin importer).

> [msg 2]
> > Another quirk that I think needs fixing:
> >
> > When I issues an import:
> >
> >  import mx.DateTime
> >
> > the whole import is handled by the importer installed at
> > the start of the import. It is not possible to install a
> > different importer e.g. in mx/__init__.py to handle the rest of
> > the import (in this case the import of subpackage DateTime). I
> > think that the importer should honor the __importer__ function
> > (this is set by imputil) if present to let it continue the import
> > of subsequent elements in the dotted name.
> 
> Sure you can. Your first importer is the "mx" importer. It has a
> dict of sub-importers. When mx/DateTime/__init__.py runs, it
> puts itself into that dict. The importer chain is now a tree.

The problem is that the special importer has to be installed
*prior* to doing the mx.DateTime import, because otherwise
the importer will not take control over the DateTime subpackage
(unless I tell it to do so explicitly, which is not really
what I want to have to do).

> This means, I think, that a "general" relative-path importer (ie,
> one that uses the default PYTHONPATH strategy), should be
> careful to install itself as the penultimate importer in the chain,
> (ie, the last before __builtin__.imp). But putting a relative-path
> search strategy into the "mx" importer is fine if it can quickly
> determine that the target is / is not a valid name in the "mx"
> namespace.

Exactly... and this brings us back to the importer list
I mentioned above.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                    98 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/