[issue29759] Deadlock in multiprocessing.pool.Pool on terminate
Michael
report at bugs.python.org
Wed Jul 5 09:08:41 EDT 2017
Michael added the comment:
I found a couple of other cases when deadlock still occurs.
1. _help_stuff_finish may remove sentinels from the queue. Some of the workers will then never get a signal to terminate.
2. worker handler thread may be terminated too late, so it may spawn new workers while terminating is in progress.
I tried to fix these two issues too in following commit: https://github.com/michael-a-cliqz/cpython/commit/3a767ee7b33a194c193e39e0f614796130568630
NB: This updated snippet has higher chances for deadlock:
"""
import logging
import multiprocessing.pool
import signal
import time
def foo(num):
return num * num
def signal_handler(signum, frame):
pass
if __name__ == '__main__':
signal.signal(signal.SIGTERM, signal_handler)
logger = multiprocessing.log_to_stderr()
logger.setLevel(logging.DEBUG)
pool = multiprocessing.pool.Pool(processes=16)
time.sleep(0.5)
pool.map_async(foo, range(16))
pool.terminate()
"""
(I am running it from dead loop in a shell script)
----------
_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29759>
_______________________________________
More information about the Python-bugs-list
mailing list