PyWart: "Python's import statement and the history of external dependencies"

[Ian Kelly]

On Fri, Nov 21, 2014 at 3:25 PM, Rick Johnson
<rantingrickjohnson at gmail.com> wrote:
>> The only exception is if you're doing "import calendar" from inside
>> the ricklib package, and you're using Python 2, and you don't have
>> "from __future__ import absolute_import" at the top of your module.
>> The solution to this is easy: just add that __future__ import to the
>> top of your module, and poof, implicit relative imports don't happen.
>> This is also fixed entirely in Python 3.
>
> I wish the irony of needing to know an "implicit rule" to
> solve an "implicit problem" could be funny, but it's really
> just sad. I can't help but be reminded of the Python zen.
> "If it's difficult to explain it's probably a bad idea".
> What's more difficult to explain than an implicit rule you
> have no knowledge of?

It's not so much implicit as an old wart that was fixed in a
backward-compatible way. As I said it's not an issue in Python 3, so
it's kind of pointless to be griping about it now.

>> > Anyone would expect that when *DIRECTLY* importing a
>> > package, if the __init__ file has code, then THAT code
>> > should be executed, HOWEVER,  not many would expect that
>> > merely "referencing" the package name (in order to import a
>> > more deeply nested package) would cause ALL the
>> > intermediate __init__ files to execute -- this is madness,
>> > and it prevents using an __init__ file as an "import hub"
>> > (without side-effects)!
>>
>> The whole point of the __init__.py file, in case you didn't intuit it
>> from the name, is to host any initialization code for the package. Why
>> on earth would you expect to import a module from a package without
>> initializing the package?
>
> Because you failed to notice that i was NOT importing the
> "package" which contained the __init__file, no, i was
> importing a *SUB PACKAGE* of the package that contained the
> __init__ file.

No, I understood that. My question stands.

> Why does the code in the main package need to run when i
> *explicitly* and *directly* fetched a "nested resource"
> within the package? Nothing within the __init__ file is
> affecting the code within the subpackage i wanted, and
> inversely, the package i wanted does not depend on anything
> within the __init__ file. There exists no symbiotic
> relationship between these two machinery, yet, by
> referencing one of them, the other starts doing unnecessary
> things!

It has nothing to do with the __init__ file specifically.  When you
import ham.spam.eggs, Python first imports ham, then ham.spam, then
ham.spam.eggs. The reason for this should be obvious: the module is
imported as a local variable in whatever context the code is running
in. "ham.spam.eggs" is not a legal variable name. Instead it has to
use the variable "ham", and populate it with the module ham. The
module "ham" then gets an attribute (i.e. module-level global) named
"spam" that is populated with the module ham.spam. Finally that module
gets an attribute "eggs" that is populated with the ham.spam.eggs
module. Because of this, it is impossible to import the module
ham.spam.eggs (under that name, anyway) without first importing ham
and ham.spam.

Another reason is that a package can affect the way its contents are
imported, e.g. by setting a package-level __path__ variable.

But also, the __init__.py is executed because it is considered to be
the initializer for the entire package. Would you expect to be able to
create a class instance and call one of its methods without having the
__init__ method called? The __init__.py file is called that because
it's meant to be analogous. What you seem to be asking for is a way
lump a subpackage into a package without treating that higher-level
package as a package. There is a way to do this, using namespace
packages, which have no __init__.py files.

> There is a workaround, but it's even more of a mess. In
> order to maintain a unique "import hub" without the chance
> of side effects from __init__ files, i can move all the code
> in "ricklib.subpkg1.__init__.py" (the code that does all the
> imports) into a normal file named
> ricklib.subpkg1._subpkg1.py".

Or you could just accept that if you don't want importing ham to
automatically import ham.spam, then you shouldn't put that import in
ham/__init__.py. Most __init__.py files *should* be empty.

> If Python expects me to use packages to protect my module
> names from clashing, but it gives me no method by which to
> import packages without causing side effects, then what is a
> boy to do (besides creating workarounds for workarounds)?

If you don't want any initialization to happen when you import your
modules, then don't put any initialization code in your __init__.py.
Or use namespace packages as mentioned above.