[issue30414] multiprocesing.Queue silently ignore messages after exc in _feeder
Antoine Pitrou
report at bugs.python.org
Fri Jun 2 12:52:01 EDT 2017
Antoine Pitrou added the comment:
Thomas, thanks for the heads up. I would suggest something like the following patch to multiprocessing.Pool:
$ git diff
diff --git a/Lib/multiprocessing/queues.py b/Lib/multiprocessing/queues.py
index 7f77837..ebbb360 100644
--- a/Lib/multiprocessing/queues.py
+++ b/Lib/multiprocessing/queues.py
@@ -260,8 +260,16 @@ class Queue(object):
info('error in queue thread: %s', e)
return
else:
- import traceback
- traceback.print_exc()
+ self._on_queue_thread_error(e)
+
+ def _on_queue_thread_error(self, e):
+ """
+ Private API called when feeding data in the background thread
+ raises an exception. For overriding by concurrent.futures.
+ """
+ import traceback
+ traceback.print_exc()
+
_sentinel = object()
Then you can write your own Queue subclass in concurrent.futures to handle that error and clean up/restart whatever needs to be cleaned up or restarted. What do you think?
----------
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30414>
_______________________________________
More information about the Python-bugs-list
mailing list