Duplicate modules problem

Chuck Esterbrook echuck at mindspring.com
Thu Feb 22 13:10:48 EST 2001


You can find a tarball with this description and accompanying source code at:
     ftp://webware.sourceforge.net/pub/webware/ModulesProb01.tar.gz

I have a problem where Python creates duplicate modules in memory rather 
than reuse the same one. For example, a module in Pkg/Mod.py ends up in 
sys.modules under keys "Pkg.Mod" and "Mod" **pointing to two distinct 
modules**.

This creates further problems: Suppose a module Foo.py contains a class 
named Foo. If the module is loaded twice as two separate instances (of 
ModuleType), then there are 2 separate Foo classes. This causes confusion 
including the failure of an assertion such as:
     assert issubclass(foo, Foo)

The Foo in that code may be pointing to the first class, while the instance 
foo may have been created from the second class. The assertion then fails!

The problem stems from the fact that Python tracks modules by a relative, 
rather than absolute, path. A simple os.chdir() or a subtley in packages 
can cause this problem.

This problem is easiest to see in an os.chdir() situation:

C:\>mkdir foo
C:\>cd foo
C:\foo>mkdir bar
C:\foo>cd bar
C:\foo\bar>echo class baz: pass > baz.py
C:\foo\bar>echo ### > __init__.py
C:\foo\bar>cd ..
C:\foo>python
ActivePython 2.0, build 202 (ActiveState Tool Corp.)
based on Python 2.0 (#8, Oct 19 2000, 11:30:05) [MSC 32 bit (Intel)] on win32
 >>> from bar.baz import baz as baz1
 >>> import os
 >>> os.chdir('bar')
 >>> from baz import baz as baz2
 >>> baz1
<class bar.baz.baz at 007908BC>
 >>> baz2
<class baz.baz at 0079075C>
 >>> baz1 == baz2
0
 >>> import sys
 >>> [mod for mod in sys.modules.items() if mod[0].count('baz')]
[('baz', <module 'baz' from 'baz.pyc'>),
  ('bar.baz', <module 'bar.baz' from 'bar\baz.pyc'>)]
 >>> mod1 = sys.modules['bar.baz']
 >>> mod2 = sys.modules['baz']
 >>> id(mod1)
136147728
 >>> id(mod2)
136147744


This problem can also be seen without any use of os.chdir(). See 
ManufactureWare/ which contains the "assert issubclass(foo, Foo)" problem 
described above.

This all applies to Python 2.0 on Windows & UNIX. I haven't tried 2.1 yet.

I believe the solution is for Python to track modules by their absolute 
path. I don't know of any other resolution to the situation other than 
modifying Python in this manner. I also don't know of any disadvantage for 
Python to track modules by absolute path.

- Does anyone know of any workarounds?
- Does anyone know why it would be bad for Python to track modules by 
absolute path?
- Is there any chance Python will fix this in the future?
- If so, by 2.1?


-Chuck

ftp://webware.sourceforge.net/pub/webware/ModulesProb01.tar.gz





More information about the Python-list mailing list