[docs] [issue16194] imp.load_dynamic imports wrong module when called several times on a multi-module .so

[Václav Šmilauer]

Václav Šmilauer added the comment:

I found the cause of the behavior (perhaps it is common knowledge, but I am new to python source); imp.load_dynamic calls the following functions

     Python/import.c: imp_load_dynamic (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l1777)
     Python/importdl.c: _PyImport_LoadDynamicModule (http://hg.python.org/cpython/file/ad51ed93377c/Python/importdl.c#l23)
     Python/import.c: _PyImport_FindExtensionObject (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l525)

where the last one uses the extensions object (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l32), which is explained at http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l449

       Magic for extension modules (built-in as well as dynamically
       loaded).  To prevent initializing an extension module more than
       once, we keep a static dictionary 'extensions' keyed by module name
       (for built-in modules) or by filename (for dynamically loaded
       modules), containing these modules. A copy of the module's
       dictionary is stored by calling _PyImport_FixupExtensionObject()
       immediately after the module initialization function succeeds.  A
       copy can be retrieved from there by calling
       _PyImport_FindExtensionObject().



The fact that extensions are keyed by file name explains why opening the .so through symlink does not return the old extension object:

     # foo.so
     # bar.so -> foo.so (works for both symlinks and hardlinks)
     imp.load_dynamic("foo","foo.so")
     imp.load_dynamic("bar","bar.so") # will return the bar module

I will investigate whether marking the module as capable of multiple initialization could be a workaround for the issue -- since the quoted comment further says (http://hg.python.org/cpython/file/ad51ed93377c/Python/import.c#l459):

       Modules which do support multiple initialization set their m_size
       field to a non-negative number (indicating the size of the
       module-specific state). They are still recorded in the extensions
       dictionary, to avoid loading shared libraries twice.

To fix the issue, I suggest that the *extensions* dict is keyed by (filename,modulename) tuple for dynamically loaded modules. This would avoid any ambiguity. Grepping through the code shows that the *extensions* object is only accessed from Python/import.c, therefore regressions should be unlikely. What do you think?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16194>
_______________________________________