[issue30414] multiprocesing.Queue silently ignore messages after exc in _feeder

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thomas, thanks for the heads up.  I would suggest something like the following patch to multiprocessing.Pool:

$ git diff
diff --git a/Lib/multiprocessing/queues.py b/Lib/multiprocessing/queues.py
index 7f77837..ebbb360 100644
--- a/Lib/multiprocessing/queues.py
+++ b/Lib/multiprocessing/queues.py
@@ -260,8 +260,16 @@ class Queue(object):
                     info('error in queue thread: %s', e)
                     return
                 else:
-                    import traceback
-                    traceback.print_exc()
+                    self._on_queue_thread_error(e)
+
+    def _on_queue_thread_error(self, e):
+        """
+        Private API called when feeding data in the background thread
+        raises an exception.  For overriding by concurrent.futures.
+        """
+        import traceback
+        traceback.print_exc()
+
 
 _sentinel = object()
 


Then you can write your own Queue subclass in concurrent.futures to handle that error and clean up/restart whatever needs to be cleaned up or restarted.  What do you think?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30414>
_______________________________________