import, functions, and threading

[Tim Peters]

[Aahz]
>> There were some posts recently discussing whether it's poor style to
>> put import statements inside functions.  I recently got reminded
>> that there's one very good reason to avoid it: Python has an import
>> lock that blocks more than one thread from executing import
>> statements.  Putting import statements inside functions that might
>> be called in a threaded environment is asking for deadlock trouble.

[Phillip J. Eby]
> If I understand this correctly, a deadlock could only occur if a lock
> was being acquired by module-level code in a newly imported module,
> after another thread acuired that lock, and then blocked waiting for
> the import lock.
>
> So, all this means is that you shouldn't acquire locks in module-level
> code, because if you do, you can run afoul of this problem entirely
> independent of whether you do any importing in functions!
>
> So, treating this as an issue with importing inside functions seems
> like a red herring.  The import lock can only be a source of deadlocks
> for module-level code, and the issue applies to *any* import
> statement.

A more common deadlock is this one, in various guises (some much subtler
than this! this is meant to be as obvious as possible), which has no explict
locks:

"""
import threading
import time

initialized = False

class Worker(threading.Thread):

    def run(self):
        global initialized
        import sha

        self.accum = sha.new()
        initialized = True
        while True:
            pass

Worker().start()
while not initialized:
    time.sleep(1)
print 'OK!'
"""

If you import that as a module, you'll deadlock:  the import lock is held by
the thread doing the import, that thread is spinning in the loop waiting for
the new thread's run() method to set initialized to True, but the new
thread's run() method is blocked on the "import sha", waiting for the import
lock.  There's no problem if sha is imported at module level instead.

The lesson I take from that isn't to avoid function-level imports, though,
it's that merely importing a module shouldn't start threads as a side
effect.  Yes, I am a Zope Corp employee <wink>.