[New-bugs-announce] [issue40444] multiprocessing.Pool deadlocks with only print statements

[Darin Tay]

New submission from Darin Tay <darin.tay at gmail.com>:

I ran into a deadlock that I've reduced to a small and consistent repro, tested on 3.7.5 and 3.8.0.


Reading through https://bugs.python.org/issue6721 now I suspect it is just another case of that, but not certain.


In particular, I'm not using any explicit threads, though presumably multiprocessing.Pool is using one under-the-hood.  If so, it seems a bit rough that multiprocessing can by itself cause the fork issues it tries to warn about ("Note that safely forking a multithreaded process is problematic.")



# This very quickly and consistently hangs after a few attempts on my machines
def run(x):
    print("Worker with ", x)
    return x

def main():
    for i in range(1000):
        print("Attempt #", i)
        from multiprocessing import Pool

        with Pool(processes=16, maxtasksperchild=1) as p:
            for entry in p.imap_unordered(run, range(50)):
                print("Main received back ", entry)


if __name__ == "__main__":
    main()

----------
components: Library (Lib)
messages: 367712
nosy: DarinTay
priority: normal
severity: normal
status: open
title: multiprocessing.Pool deadlocks with only print statements
type: behavior
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40444>
_______________________________________