[issue20660] Starting a second multiprocessing.Manager causes INCREF on all object created by the first one.

[Richard Oudkerk]

Richard Oudkerk added the comment:

On Unix, using the fork start method (which was the only option till 3.4), every sub process will incref every shared object for which its parent has a reference.

This is deliberate because there is not really any way to know which shared objects a subprocess might use.  (On Windows where only things pickled as part of the process object are inherited by the child process, we can know exactly which shared objects the child process should incref.)

Typical programs will only have a single manager (or a very small number) but may have a large number of normal processes (which will also do the increfing).  I do not think that this is worth trying to fix, particularly as it can cause compatibility problems.

For 3.4 you can use the "spawn" or "forkserver" start methods instead.

import multiprocessing, logging

objs = []

def newman(n=50):
    m = multiprocessing.Manager()
    print('created')
    for i in range(n):
        objs.append(m.Value('i',i))
    return m

def foo():
    pass

if __name__ == '__main__':
    ## Try uncommenting next line with Python 3.4
    # multiprocessing.set_start_method('spawn')
    multiprocessing.log_to_stderr(logging.DEBUG)
    print('#### first man')
    m1 = newman()
    print('#### starting foo')
    p = multiprocessing.Process(target=foo)
    p.start()
    p.join()

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20660>
_______________________________________