[Python-Dev] Import lock knowledge required!

[Guido van Rossum]

[MAL]
> I wonder whether a general lock such as the one used in import
> is such a good idea. Perhaps it should only lock the importing
> of a specific module, keeping the locks in a dictionary indexed by
> module name instead of a static C variable ?!

I've been thinking along the same lines.  We could make the import
locking much finer-grained, and limit the blocking only to threads
that are importing a module that is in the middle of being loaded by
some other thread.

But this is hard work, and I suggest that we put this off until Python
2.4 so we can do it right.

> Then again it's hard to know the real name of the module being
> searched before finding it...

There could be a short-lived lock for that problem.

> I see a more general problem here: the lock prevent starting
> up threaded applications which use client-server logic between
> the threads. If the application's main thread starts a client
> thread as a result of an import which then tries to import
> other Python modules, you have a deadlock. (At least that's how
> I understand the current implementation.)

Correct.

> Don't know about others, but I frequently use the idiom of
> placing the server's main code in a separate module and then
> have small startup script importing this module. This kind
> of setup is also advertised for CGI programs, so it may not
> be uncommon out there.

We had this problem with Zope2 -- I don't know why the import lock
didn't bite us before, but we decided to change the Zope startup code
so that you have to import Zope first and then, separately, make a
call to start it.  You could do the same for your application, and I
recommend that Mark does the same for his.

--Guido van Rossum (home page: http://www.python.org/~guido/)