[Chicago] Watch out for Queue on cygwin
Phil Robare
verisimilidude at gmail.com
Tue Oct 7 23:38:24 CEST 2008
I created a simple threaded app - control thread, network thread,
output thread and communication between the threads was done using the
synchronized queue from the Queue module. After I got it working it
would run for a while and then crash with:
sem_init: Resource temporarily unavailable
Error while running: (<class 'thread.error'>, error("can't allocate
lock",), <traceback object at 0x7ff41eb4>)
File "SoupReader.py", line 252, in run
line=self.in_queue.get(timeout=1.0)
File "/tmp/python.6884/usr/lib/python2.5/Queue.py", line 174, in get
self.not_empty.wait(remaining)
File "/tmp/python.6884/usr/lib/python2.5/threading.py", line 208, in wait
waiter = _allocate_lock()
Shutting down on error...
After spending time with the Queue.py and threading.py source (and
quickly figuring out my problem is in the C code) I googled sem_init,
and found it is the low-level Unix API for semaphore creation. Since
I am forced by work situation to use Windows I was actually running
Python under cygwin. Googling sem_init and cygwin found this post
from 2002 on the Python-Dev mailing list:
http://mail.python.org/pipermail/python-dev/2002-February/020492.html
The discussion there is that cygwin's sem_init, as called in the
Python libraries, in inherently subject to a race condition. And that
fits the symptoms I was seeing. Running the real windows python I
have no problems (other than those inherent in using Idle rather than
my preferred Vim + command line).
As I am unsure what to do with this hard won knowledge I have decided
to post it here, spamming it out to the teeming millions on this list
that don't care in the hope that I can save someone the pain I have
gone through.
Phil
More information about the Chicago
mailing list