[Python-Dev] Import hassle

[Martin Sjögren]

Hello

I've been writing quite a few mails lately, all concerning import
problems. I thought I'd write a little longer mail to explain what I'm
doing and what I find strange here.

Basically all (at least the 10-20 ones I've checked) the C modules in the
distribution have one thing in common: if something in their initFoo()
function fails, they return without freeing any memory. I.e. they return
an incomplete module.

The only way I can think of that one of the standard modules could fail i=
s
when you're out of memory, and that's kinda hard to simulate, so I put in
a faked failure, i.e. I raised an exception and returned prematurely (in
one of my own C modules, not one in the distribution!).

The code looks like this:
    PyErr_SetString(PyExc_ImportError, "foo");
    return;
    /* do other things here, this "fails" */

>>> import Foo
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: foo
>>> import Foo
>>> dir()
['Foo', '__builtins__', '__doc__', '__name__']

Huh?! How did this happen? What is Foo doing there?

Even more interesting, say that I create a submodule and throw in a bunch
of PyCFunctions in it (I stole the code from InitModule since I don't kno=
w
how to fake submodules in a C module in another way, is there a way?). I
create the module, fail on inserting it into the dictionary and DECREF it.
Now, that ought to free the darn submodule, doesn't it? Anyway, I wrote a
simple "mean" script to test this:

try: import Foo
except: import Foo
while 1:
  try: reload(Foo)
  except: pass

And this leaks memory like I-don't-know-what!
What memory doesn't get freed?


Now to my questions: What exactly SHOULD I do when loading my module fail=
s
halfway through? Common sense says I should free the memory I've used and
the module object ought to be unusable.

Why-oh-why can I import Foo, catch the exception, import it again and it
shows up in the dictionary? What's the purpose of this?

How do I work with submodules in a C module?


I find the import semantics really weird here, something is not quite
right...

Regards,
Martin Sj=F6gren

--=20
Martin Sj=F6gren
  martin@strakt.com              ICQ : 41245059
  Phone: +46 (0)31 405242        Cell: +46 (0)739 169191
  GPG key: http://www.strakt.com/~martin/gpg.html