import packet.module without importing packet.__init__ ?

[Gelonida N]

Hi Steven,

Thanks for your answer.

On 09/11/2011 02:56 AM, Steven D'Aprano wrote:
> Gelonida N wrote: 
>> Is it possible to import a module from a packet without importing its
>> __init__.py ?
> 
> Untested, but I think so. But you shouldn't. The solution (if it does work,
> as I said I haven't tested it) is a hack.
> 
> Suppose you have a library like this:
. . .
. . .
> The directory "modules" is in your Python path, so you can "import spam"
> or "import package". The trick is to add "modules/package" to the Python
> path so that "import cheese" will find cheese.py directly.
> 

Yes it's a hack, but good to know it exists. Just in case.

> Another hack would be to add a hard link to the top level:
> 

As you said: Hard links would be a little annoying on some file systems.

> 
> But generally, you should design your package so that it doesn't matter
> whether or not __init__.py is imported first. I see three reasnable
> situations:
> 
> (1) package.cheese should rely on package, in which case it requires
> package.__init__ to be imported first. (Sometimes I put code common to the
> entire package in __init__.py.)
> 
> (2) Just don't worry about the fact that package.__init__ gets imported
> first. Do you have any idea how many modules get imported when Python
> starts up? What's one more?
> 
> (3) If cheese.py truly is independent of package, then it shouldn't be part
> of package. Just make it a stand alone module outside the package
> directory, and be done. (Your installer can still treat cheese.py as a
> dependency of package, and ensure that they are both installed.)
> 


There's still something, that I am philosophycally missing.

Wy do I have to import the entire tree if I'm just interested in a leave.

this is not always desired.

let's imagine

kitchen.__init__
kitchen.knives
kitchen.pans
kitchen.pots


or alternatively
os
os.path
os.wheteverelse

Let's assume, that  kitchen/__init__.py is importing the sub modules
knives, pans and pots

then I understand of course, that the next lines would import the entire
kitchen

> import kitchen # will import all activities
> kitchen.pots.boil(egg)


But if I know that I just want to boil some eggs I would really
appreciate if I could just say

> from kitchen.pot import boil
> boil(eggs)
without having the side effect of getting pans and knives also loaded
into memory.

Moving pots out into a separate package doesn't really feel right.
I agree, that a 10 level deep package tree is not really desirable, but
having every package flat on the top level doesn't sound that good either.

So it seems, that developers of huge packages shouldn't really do a lot
in __init__.py.

Otherwise any user would always 'suck in' the entiry package with all
its sub modules.




To give more details about why I asked this question:

The real setup looks a little different and there are some issues, that
I am having:

One package in question is a huge django application.

Within the package is one module defining some constants and tiny
functions, which I would like to reuse from some command line tools,
which should start quickly without loading any of the django specific
modules. (the startup penalty and the memory overhead would be noticable)

I am using rpc4django, which expects, that __init__.py of the django
application exports some all rpc functions
which will basically import 95% of the django application and the entire
django frame work (none of which were required by my command tool,
support utility for this application)


I could of course create a separate package just for this tiny sub
module, but somehow it doesn't feel right to me.


The second issue is a little more complicated. basically I end up having
circular dependencies.

as I have two applications that  send a few customs django signals forth
an back.

If I wouldn't import __init__.py or if I moved the declaration of the
signals into a third package then circular dependencies would disappear.


So it seems, that if I can't avoid loading __init__.py in
a nice way I have to create  one or two tiny packages outside of my
django application,
or change the python path  as you proposed (increases of course name
space clashes)

or to find a way for rpc4django of loading / locating the rpc functions
without putting them  in __init__.py change the rpc functions such, that
they will only import the other sub modules when being called the first
time (proxies / delayed imports, . . .)