Python's import: why doesn't it work?

[Jurie Horneman]

Once again my attempt to profit from Python's vaunted productivity
increase has come to a screeching halt because some innocent thing
that one would reasonably expect to work in one line, intuitively, and
without even having to think about, turns into a nightmare already
lasting more than an hour, requiring numerous experiments, searching
through the documentation - both manually and using my editor's
find-in-files - finally leaving me no other option but to write a post
to comp.lang.python, knowing that my programming flow for today is
well and truly gone. I hate doing this, because I know I cannot keep
my frustration out of my writing, and also that someone will reply and
gently or not so gently point out something which actually IS written
down in some obscure corner of the documentation, despite my attemps
to RTFM.

So, what is it this time? No, it's not Unicode, another aspect that
frequently trips me up, causing my face to meet the rough asphalt of
the infobahn. It's importing modules. Again.

(This is an obscure reference to an attempt on my part to implement
MBCS conversion functions as Python codecs. This failed miserably and
despite the helpful advice I received here, I never did try again. Too
bad, because those conversion functions are pretty good. And since I'm
doing some parenthesized ranting here: does anyone outside of the
Python world use 'codec' to refer to anything except audio / video
codecs? I actually had to post here just to find out that what I was
looking for were *called* codecs. Sheesh.)

Anyway.

I have this script, let's call it 'blah.py'. It sits in a directory
far, far away from where my Python is installed. It wants to include a
module called 'intro' from a subdirectory called 'templates'.

This is evidently impossible.

Among my vain, futile attempts were:

*** Solution 1 ***
The World is a Beautiful Place

>>> import templates.intro

Ha! ha! ha! OK, even I knew this was a bit optimistic.

*** Solution 2 ***
.PTH file

Put a templates.pth containing the single line 'templates' in the same
place as 'blah.py', then

>>> import templates.intro
or
>>> import intro

No dice: 'ImportError: No module named...' etc.

*** Solution 3 ***
IMPorting by hand

>>> import imp
>>> f = open('templates\intro.py')

>>> imp.find_module('intro.py')
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No module named intro.py

>>> imp.find_module('intro')
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No module named intro

>>> imp.find_module('intro', '\\templates\\')
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No frozen submodule named .\templates\.intro

(No WHAT?)

>>> imp.find_module('intro', '\\templates\\')
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No frozen submodule named \templates\.intro

>>> imp.find_module('intro', ['\\templates\\'])
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No module named intro

>>> imp.find_module('intro.py', ['\\templates\\'])
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No module named intro.py

>>> imp.find_module('intro', ['.\\templates\\'])
(<open file '.\templates\intro.py', mode 'r' at 0x01511110>,
'.\\templates\\intro.py', ('.py', 'r', 1))

Whoa! I finally managed to call imp.find_module without raising an
exception! OK, now to call imp.load_module, because the documentation
says:

"Load a module that was previously found by find_module()"

Alright, so there seems to be some connection between find_module and
load_module... The latter probably takes the return value of the
former as a parameter, right? Right? Wrong.

Here are my attemps to call imp.load_module:

x = imp.find_module('intro', ['.\\templates\\'])

>>> y = imp.load_module('intro', f, 'intro.py', x)
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
TypeError: load_module() argument 4, item 0 must be string, not file

>>> y = imp.load_module('intro', x)
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
TypeError: load_module() takes exactly 4 arguments (2 given)

Okay, wait, my assumption was obviously naive, better read the doc...
aha. Almost the same parameters as find_module... wonder why I even
needed to call find_module in the first place. OK:

>>> y = imp.load_module('intro', f, 'intro.py', imp.get_suffixes())
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
TypeError: load_module() argument 4 must be sequence of length 3, not
5

What? But... but... *splutter* the doc says:

"The description argument is a tuple, as would be returned by
get_suffixes(), describing what kind of module must be loaded."

So why doesn't this work? OK, I have no idea why I'm bothering with
these kinds of arcana, but for the heck of it I'll try:

>>> y = imp.load_module('intro', f, 'intro.py', ('.py', 'r', 1))
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "intro.py", line 29, in ?
ImportError: No module named Base

Yay! What?! Now my IMPORTED module can't find the modules IT is trying
to import? (It doesn't matter if I put Base next to Blah.py or in
templates - Intro can't find it.)

Arrrgghhh!!!! OK, enough time spent on this dead end. (Oh, and by the
way not all the functions of the imp module are documented.)

*** Solution 4 ***
Packages

OK, more searching, browsing and scouring my hard drive for the word
'import'. Of course! I forgot that in the Python docs, important
information is cunningly hidden in the Tutorial section! (Really, am I
being unreasonable for assuming that once one has reached a certain
level of knowledge, one shuns the Tutorial and goes straight for the
main documentation, because that's where one can find everything there
is to know? That's only how it works in every language or piece of
software in general I've ever encountered.)

So there are these things called 'packages'. Wow! Handy, that! All one
needs to do is put an empty file called '__init__.py' in a directory,
and hey! That directory becomes a package and Bob's your uncle.

So I put an empty file called '__init__.py' in 'templates' and try the
following:

>>> import templates.intro
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No module named templates.intro

>>> from templates import intro
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
ImportError: No module named templates

This is *exactly* how it's written in the Tutorial, and
it-does-not-work.

*** Solution 5 ***
Abject Defeat, Ask on Internet, Be Ridiculed by Peers for Not Managing
Basic Operation

Amusing little anecdote: I've performed all of the above steps twice -
once while growing increasingly frustrated, and a second time while
writing this post. Twice, while typing in the code above, I at some
point entered:

import intro

... and it would work. For real. The symbols were there. Not a peep
out of the interpreter.

Because I don't trust PythonWin (don't get me started on PythonWin, I
haven't found a better IDE yet, I may actually succumb and PAY for one
soon *shock horror*) I quit and restarted and lo and behold! the
interpreter complained that there was no such module, as it had been
doing all along, the horrible beast.

In other words, at random, unreproducable moments, Python perversely
imported my module, whereas normally it refused to.

To cut a long story... hm... Please please help me! Stop me before I
whine again. And please don't be offended by my ranting, I just can't
hide this much frustration. I'm using Python 2.2 by the way.

Thanks in advance,

Jurie.