[Python-Dev] PEP 366 - Relative imports from main modules

[Nick Coghlan]

Brett Cannon wrote:
> On 7/8/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> As with the current ``__name__`` attribute, setting ``__package__`` will
>> be the responsibility of the PEP 302 loader used to import a module.
>> Loaders which use ``imp.new_module()`` to create the module object will
>> have the new attribute set automatically to
>> ``__name__.rpartition('.')[0]``.
> 
> Is this really the best semantics for this?  Let's say I have
> A/B/__init__.py and A/B/C.py.  With these semantics I would have A.B
> having __package__ be 'A' and A.B.C having 'A.B'.
> 
> While I agree that the A.B.C setting is correct, is the A.B value what
> is truly desired?  Is an __init__ module really to be considered part
> of the above package?  I always viewed the __init__ module as part of
> its own package.  Thus I expected A.B to have __package__ set to
> 'A.B'.

Good point - PEP 328 makes it explicit that __init__.py is treated like 
any other module in the package for purposes of relative imports, so the 
semantics you suggest are the ones required. I hadn't actually thought 
about this case, as it wasn't relevant when the new attribute applied 
only to the main module.

However, those semantics mean that we won't be able to automatically add 
the new attribute inside imp.new_module(), as that function doesn't know 
whether or not the new module is a package.

> Beyond just what I expected, the reason I bring this up is that if
> __package__ had the semantics I am suggesting, it is trivial to
> discover what modules are the package __init__ modules (as
> ``__package__ == __name__``) compared to being a submodule
> (``__package__ and __package__ != __name__``).  As of right now you
> can only do that if you examine __file__ for __init__.py(c), but that
> is highly dependent on how the module was loaded.  It might be nice if
> what kind of module (top-level, package, or submodule) something is
> based on its metadata. 

This part of the argument isn't relevant though, as it's already trivial 
to determine whether or not a module is a package by checking for a 
__path__ attribute. That's what PEP 302 specifies, and it is how the 
relative import machinery currently determines whether to use __name__ 
or __name__.rpartition('.')[0] as the base for relative imports.

Given the above limitations, I propose that we document the new 
attribute as follows:

"If the module global __package__ exists when executing an import 
statement, it is used to determine the base for relative imports, 
instead of the __name__ and __path__ attributes. This attribute may be 
set by the interpreter before a module is executed - whether or not it 
is set automatically in a given module is implementation dependent."

And for the CPython implementation, I propose that we set the new attribute:
   1. When the main module is executed using the -m switch
   2. When a relative import is executed and the attribute is not yet set

This will allow any module which uses relative imports to benefit from 
the micro-optimisation of caching the package name in normal modules 
(regardless of how the module gets loaded), as well as allowing relative 
imports from the main module (which is the main goal of the PEP).

With the way PEP 302 hands off creation of the module and execution of 
its code to the loader objects, I don't see any way to guarantee that 
__package__ will always be set - this seems like a reasonable compromise.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org