[issue6653] Potential memory leak in multiprocessing

[Jesse Noller]

Jesse Noller <jnoller at gmail.com> added the comment:

Additional comments from Sturla:

Hello Jesse,

Yes there is a bug in multiprocessing.

Diagnosis:

- Processes created by multiprocessing (mp.Process or mp.Pool) exit in a 
way that prevents the Python interpreter from running deallocation code 
for all extension objects (only the locals are cleaned up). Resources 
allocated by extension objects referenced in globals may leak 
permanently.

Sub-processes seem to commit an ungraceful suicide on exit. If the 
kernel cleans up after a non-graceful exit this is ok. But if the kernel 
do not, as in the case of System V IPC objects, we have a permanent 
resource leak. This is very similar to the reason why manually killing 
threads is prohibited in Python.

I have example code to show this. It creates a system-wide memory leak 
on Linux/Unix (present until the next reboot), unless the last statement 
in the target of mp.Process ensures a manual clean up of the globals.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6653>
_______________________________________