[issue38501] multiprocessing.Pool hangs atexit (and garbage collection sometimes)

[Eric Larson]

Eric Larson <larson.eric.d at gmail.com> added the comment:

If that's out of contract, perhaps there should probably a big, visible warning at the top of the multiprocessning docs stating that creating one of these objects requires either using a context manager or ensuring manual `.close()`ing?

1. It either used to be in contract not to close it manually or was wrongly represented, the first Python 2.7 example in the docs (https://docs.python.org/2/library/multiprocessing.html#introduction) is:

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    p = Pool(5)
    print(p.map(f, [1, 2, 3]))

So I suspect this (difficult to track down) problem might hit users without more adequate warning.


2. I'm surprised it's actually out of contract when the 3.8 docs state that close will be automatically called:

> close()
>
> Indicate that no more data will be put on this queue by the current process. The background thread will quit once it has flushed all buffered data to the pipe. This is called automatically when the queue is garbage collected.

and

> terminate()
>
> Stops the worker processes immediately without completing outstanding work. When the pool object is garbage collected terminate() will be called immediately.

Or perhaps I misunderstand what this is saying?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38501>
_______________________________________