[issue8428] buildbot: test_multiprocessing timeout (test_notify_all? test_pool_worker_lifetime?)

[Charles-Francois Natali]

Charles-Francois Natali <neologix at free.fr> added the comment:

Attached is a patch fixing this race, and a similar one in Pool's terminate.

----------
keywords: +patch
Added file: http://bugs.python.org/file21608/pool_shutdown_race.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue8428>
_______________________________________
-------------- next part --------------
diff -r bbfc65d05588 Lib/multiprocessing/pool.py
--- a/Lib/multiprocessing/pool.py	Thu Apr 07 10:48:29 2011 -0400
+++ b/Lib/multiprocessing/pool.py	Sun Apr 10 23:52:22 2011 +0200
@@ -322,6 +322,8 @@
         while pool._worker_handler._state == RUN and pool._state == RUN:
             pool._maintain_pool()
             time.sleep(0.1)
+        # send sentinel to stop workers
+        pool._taskqueue.put(None)
         debug('worker handler exiting')
 
     @staticmethod
@@ -440,7 +442,6 @@
         if self._state == RUN:
             self._state = CLOSE
             self._worker_handler._state = CLOSE
-            self._taskqueue.put(None)
 
     def terminate(self):
         debug('terminating pool')
@@ -474,7 +475,6 @@
 
         worker_handler._state = TERMINATE
         task_handler._state = TERMINATE
-        taskqueue.put(None)                 # sentinel
 
         debug('helping task handler/workers to finish')
         cls._help_stuff_finish(inqueue, task_handler, len(pool))
@@ -484,6 +484,11 @@
         result_handler._state = TERMINATE
         outqueue.put(None)                  # sentinel
 
+        # we must wait for the worker handler to exit before terminating
+        # workers because we don't want workers to be restarted behind our back 
+        debug('joining worker handler')
+        worker_handler.join()
+
         # Terminate workers which haven't already finished.
         if pool and hasattr(pool[0], 'terminate'):
             debug('terminating workers')