[Import-SIG] New PEP draft: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 09:28 AM 8/16/2011 +1000, Nick Coghlan wrote:
>I'm actually not sure this is a viable approach as currently described
>in the PEP - most of the existing import machinery assumes that parent
>modules will exist in sys.modules before child modules are imported.

That assumption isn't being violated as much as you might think.  The 
PEP merely requires that one hold off on creating the parent module 
instance until you're just about to load the child.  This is quite 
straightforward to implement if you process the import iteratively 
from left to right, as CPython (pre-importlib) does.

In effect, the algorithm is:

    path = sys.path

    for each part of the name:
        check for an already imported name up to this point
        if it's already imported:
             path = module.__path__
        else:
             try to find the module (using 'path')
             if the module is found:
                 add any missing parent modules to sys.modules
                 load the module
                 path = module.__path__
             else:
                 path = virtual path for the missing module

    if we have a module:
        return it
    else:
        raise ImportError

Very simple, really.  Granted, the "fill in any missing parent 
modules" is a wee bit tricky and reintroduces recursion into the 
mix.  (See http://pastebin.com/6e29v8LR for the full details, 
including a draft implementation of an auto-updating proxy __path__ object.)

But the only place where the "parent modules are already in 
sys.modules" assumption can be broken here is in find_module() calls 
-- *not* in load_module() or any actual module code.  And this 
assumption is only broken in scenarios where, in today's Python, the 
import would already have failed first.