[Import-SIG] pkgutil.extend_path

[Eric V. Smith]

On 05/15/2012 06:04 PM, PJ Eby wrote:
> On Tue, May 15, 2012 at 1:26 PM, Eric V. Smith <eric at trueblade.com
> <mailto:eric at trueblade.com>> wrote:
> 
>     I'm looking at fixing pkgutil.extend_path in order to support namespace
>     packages where some portions use PEP-420 and some use extend_path.
> 
>     The first thing I notice is that there are no tests for
>     pkgutil.extend_path :(
> 
>     extend_path currently just examines the filesystem directly, which means
>     it doesn't support portions in zip files or other finder/loaders.
> 
>     But if I understand PJE and others correctly, the idea is to modify
>     extend_path so it calls the path_hook finders instead of looking at the
>     filesystem (in order to find the other __path__ entries). 
> 
> 
> Well, I actually wasn't trying to support that at all; I've never used
> extend_path() myself, so I was only talking about implementing
> transitional support for the PEP in declare_namespace().

I'm mostly interested in extend_path() because:
a) it's in the standard library
b) if it can be made to work, I assume declare_namespace() can, too

> That being said, it sounds like adding support in extend_path() might
> also be worthwhile.  Presumably, it would *not* auto-update, since
> extend_path() doesn't currently do that.  (Just as declare_namespace()
> *should* auto-update, because it currently does.)

Right. I have extend_path() working with mixed namespace packages: those
that have an __init__.py with extend_path, and those with no
__init__.py. In this case, it does not support auto-updating (although
it easily could, if we expose that from _bootstrap). I'll update the PEP
to mention returning the portions iterable even if a loader is returned,
clean up the code, and check it in.

I'll leave declare_namespace() for someone else. I might have time to
look at it if/when the PEP is accepted, but no promises.

>     There's the added issue of how to deal with .pkg files. Is only
>     supporting them from the filesystem okay? Or is it worth the hassle of
>     creating some finder API to access them?
> 
> 
> I don't know who actually uses them, but then I don't know who uses
> extend_path(), period.  The only examples I was able to find with Google
> and Nullege are of the form "try: declare_namespace() except:
> extend_path()" -- that is, code that first tries to use
> declare_namespace() instead.

I haven't touched the .pkg code. It doesn't work terribly well if a
zipimporter is used, but then it never did. At least I'm not making it
worse.

Once this is checked in, I think the work on the PEP and sample
implementation is done.

Eric.