race/deadlock when creating a multiprocessing.manager instance while importing a module ?

[Dave Angel]

Sebastien Binet wrote:
> hi there,
>
> say I have this module
>
> ## farnsworth ##
> __all__ = [
>     'mgr',
>     'maths',
>     ]
>
> from multiprocessing.managers import BaseManager
>
> class MathsClass(object):
>     def add(self, x, y):
>         return x + y
>     def mul(self, x, y):
>         return x * y
>
> class MyManager(BaseManager):
>     pass
>
> MyManager.register('Maths', MathsClass)
>
> def _setup():
>     print "creating a manager..."
>     mgr = MyManager()
>     print "starting the manager..."
>     mgr.start()
>     print "sciencing faster..."
>     maths = mgr.Maths()
>     print maths.add(4,3)
>     print maths.mul(7,8)
>     print "done with sciencing."
>     return (mgr, maths)
>
> # exec at module import
> mgr, maths = _setup()
>
> # prevent hysteresis + clean-up
> del _setup
> ## EOF ##
>
> if I use it like so:
>
> $ python -m farnsworth
> creating a manager...
> starting the manager...
> sciencing faster...
> 7
> 56
> done with sciencing.
>
> all is fine, but if I try to use it thru an import:
> $ python
> py> import farnsworth.mgr as mgr
> creating a manager...
> starting the manager...
> sciencing faster...
>
> [stuck for some time... hitting ^C]
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "farnsworth.py", line 32, in <module>
>     mgr, maths = _setup()
>   File "farnsworth.py", line 25, in _setup
>     maths = mgr.Maths()
>   File "/usr/lib/python2.6/multiprocessing/managers.py", line 634, in
> temp
>     token, exp = self._create(typeid, *args, **kwds)
>   File "/usr/lib/python2.6/multiprocessing/managers.py", line 532, in
> _create
>     conn = self._Client(self._address, authkey=self._authkey)
>   File "/usr/lib/python2.6/multiprocessing/connection.py", line 140,
> in Client
>     answer_challenge(c, authkey)
>   File "/usr/lib/python2.6/multiprocessing/connection.py", line 372,
> in answer_challenge
>     message = connection.recv_bytes(256)         # reject large
> message
> KeyboardInterrupt
>
> is this a known limitation/feature of the multiprocessing module ?
> is there a workaround (acquiring some import lock maybe) ?
>
> cheers,
> sebastien.
>
> PS:
> $ python
> Python 2.6.4 (r264:75706, Oct 27 2009, 06:25:13)
> [GCC 4.4.1] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>
> $ python -c 'import sys; print sys.version_info'
> (2, 6, 4, 'final', 0)
>
> $ uname -a
> Linux farnsworth 2.6.31-ARCH #1 SMP PREEMPT Tue Nov 10 19:01:40 CET
> 2009 x86_64 Intel(R) Core(TM)2 Duo CPU T9400 @ 2.53GHz GenuineIntel
> GNU/Linux
>
>   
Since I don't see any other responses, I'll give my guess, even though 
I'm not very experienced with the multiprocessing module.

It's my understanding that threads may not be created or destroyed 
during an import.  So you need to find a way to defer the creation till 
the imports are done.

DaveA