[Python-Dev] Import lock knowledge required!

[eric jones]

> [MAL]
> > I wonder whether a general lock such as the one used in import
> > is such a good idea. Perhaps it should only lock the importing
> > of a specific module, keeping the locks in a dictionary indexed by
> > module name instead of a static C variable ?!

I am interested in this also.

> 
> [Guido]
> I've been thinking along the same lines.  We could make the import
> locking much finer-grained, and limit the blocking only to threads
> that are importing a module that is in the middle of being loaded by
> some other thread.
> 
> But this is hard work, and I suggest that we put this off until Python
> 2.4 so we can do it right.

I ran into this exact thing when trying to get wxPython windows (data
plots) to co-exist peacefully on the screen in parallel with an
interactive shell.  The module is called gui_thread, and we use it in
SciPy.

http://www.scipy.org/site_content/tutorials/gui_thread
  
The command line took the main thread, and a wxPython app was started in
the background thread.  The first import of wxPython had to occur in the
background thread for wxPython's to be happy.  Initially, I put a lock
in the import of gui_thread which waited for the wxPython import to
complete before allowing the gui_thread import to complete.  If 'import
gui_thread' is the first statement executed, this guarantees that
wxPython is always imported first in the background thread.
Unfortunately, it also causes deadlock because of the import lock.

A discussion that occurred about this problem on the thread-sig with is
summarized here:

http://www.scipy.org/site_content/tutorials/import_thread_lock_discussio
n

It includes an alternative version of several import.c routines
(compared to 1.5.2 I believe) that solved my problem as proposed by MAL
above and passed all the regression tests at the time.  I just plugged
the code into the 2.3CVS, and it mainly works.  But, there are some
import related errors in the regression tests for: 

test_threaded_import
test_loggingllbacks


I could spend some time on these if there is interest in getting this in
the current release.  Also, this code would need review by someone that
is an expert on the subtleties of the import code to make sure it is
sound.  Let me know, and I'll submit a patch with of the current code.

eric