[Distutils] Re: new import mechanism and a "small" distribution

[Just van Rossum]

At 10:47 AM -0800 1/31/99, Greg Stein wrote:
>By "loader", I will presume that you mean an instance of an Importer
>subclass that is defining get_code(). Here is the method as defined by
>imputil.Importer:
>
>  def get_code(self, parent, modname, fqname):

I see in your code that you build a chain of import hooks that basically
work like a list of loaders. Why don't you use sys.path for this? (Current
implementation details, probably?)

I'd suggest a loader object should be a callable, ie. get_code() should be
called __call__()... Changing your pseudo code from the distutils-sig to:

  for pathentry in sys.path:
    if type(pathentry) == StringType:
      module = old_import(pathentry, modname)
    else:
      module = pathentry(modname)  # <--
    if module:
      return module
  else:
    raise ImportError, modname + " not found."

It's the only public method. Makes more sense to me. It also seems that it
would make it easier to implement loaders (and the above loop) in C.

>That method is the sole interface used by Importer subclasses. To define
>a custom import mechanism, you would just derive from imputil.Importer
>and override that one method.
>
>I'm not sure if that answers your question, however. Please let me know
>if something is unclear so that I can correct the docstring.

The only thing I find unclear is when the parent argument should be used or
not. Is it only for importing submodules?

>Oh, geez. And I totally spaced on one feature of imputil.py. There is a
>way to define an import mechanism for very simple uses. I created a
>subclass named FuncImporter that delegates the get_code() method to a
>user-supplied function. This allows a user to do something like this:
>
>import imputil
>
>def get_code(parent, modname, fqname):
>  ...do something...
>
>imputil.install_with(get_code)

Cool! Another reason to allow any callable object in sys.path...

>The install_with() utility simply creates a FuncImporter with the
>specified function and then installs the importer. No need to mess with
>subclasses.
>
>> - how are packages identified
>
>If get_code() determines that the requested module is a package, then it
>should return the integer 1 along with the code object for that
>package's module. In the standard package system, the code object is
>loaded from __init__.py.
>
>An example: let's say that get_code() is called with (None, "mypkg",
>"mypkg") for its arguments. get_code() finds "mypkg" in whatever path it
>is configured for, and then determines that it represents a directory.
>It looks in the directory for __init__.py or .pyc. If it finds it, then
>mypkg is a real package. It loads code from __init__.py and returns (1,
>code). The Importer superclass will create a new module for "mypkg" and
>execute the code within it, and then label it as a package (for future
>reference).
>
>Internally, packages are labelled with a module-level name: __ispkg__.
>That is set to 0 or 1 accordingly.

Right now a package is labeled with a __path__ variable. If it's there,
it's a package. Is it neccesary to define something different?

>The Importer that actually imports a
>module places itself into the module with "__importer__ = self". This
>latter variable is to allow Importers to only work on modules that they
>imported themselves, and helps with identifying the context of an
>importer (whether an import is being performed by a module within a
>package, or not).

Ok, so __importer__ is there instead of __path__. That makes sense. The
current __path__ variable makes it look like it can have its own
sys.path-like thingy, but that's not quite what it is (?). It is even
abused: if it is a string, it is supposed to be the (sub)package's full
name and means that any submodule is to be located like a normal module,
but with the full name (eg. mypkg.sub1.foo). This is crucial for freezing
and handy in other cases (it allows submodules to be builtin!). While this
is cool and handy, it smells really funny. As a case study, could you show
how this stuff should work with the new import mechanism?

Just