[New-bugs-announce] [issue29861] multiprocessing Pool keeps objects (tasks, args, results) alive too long

[Antoine Pitrou]

New submission from Antoine Pitrou:

The various workers in multiprocessing.Pool keep a reference to the last encountered task or task result.  This means some data may be kept alive even after the caller is done with them, as long as some other task doesn't clobber the relevant variables.

Specifically, Pool._handle_tasks(), Pool._handle_results() and the toplevel worker() function fail to clear references at the end of each loop.

Originally reported at https://github.com/dask/distributed/issues/956

----------
components: Library (Lib)
messages: 289894
nosy: davin, pitrou, sbt
priority: normal
severity: normal
stage: needs patch
status: open
title: multiprocessing Pool keeps objects (tasks, args, results) alive too long
type: resource usage
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29861>
_______________________________________