[issue12738] Bug in multiprocessing.JoinableQueue() implementation on Ubuntu 11.04

[Michael Hall]

New submission from Michael Hall <michaelhall1k at gmail.com>:

I recently switched to Ubuntu 11.04 from OpenSUSE 11.4, and when I go to run a project I coded a couple days ago under OpenSUSE using the multiprocessing library, it hangs when it did not under OpenSUSE.

Specifically, I am using two queues, work_queue from which the children get jobs, and results_queue where they place their results before calling JoinableQueue.task_done() and grabbing the next result. I use the "poison pill" technique to terminate the children, where a None object is placed at the end of the queue for each child, and when they get one of the terminating objects they call task_done() again (to account for the None object) and exit.

In the main process, after spawning all of the children (one per physical CPU), it joins with the work_queue in order to wait for all of its children to finish.

This is pretty much a cookie-cutter multiprocessing implementation that I've used successfully for years under OpenSUSE, but for some odd reason the exact same code does not work under Ubuntu.

I would try porting to python 3.x, but the rest of my research team is still using 2.7, so that's not really an option right now.

----------
components: Library (Lib)
messages: 141932
nosy: Michael.Hall
priority: normal
severity: normal
status: open
title: Bug in multiprocessing.JoinableQueue() implementation on Ubuntu 11.04
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12738>
_______________________________________