[issue3419] multiprocessing module is racy

[Mark Dickinson]

Mark Dickinson <dickinsm at gmail.com> added the comment:

Here's what's going on with the incref error:

Look in the Server class, in managers.py:  consider a shared object with 
id 'id'.  When a reference to the shared object is created, its id is 
added to the id_to_refcount dictionary:

{id: None}

and *shortly afterwards*, the refcount is incremented to get:

{id: 1}

When the object is deleted or goes out of scope later on, the refcount 
is decref'd, and id is removed entirely from id_to_refcount.

The problem occurs when there are two processes simultaneously asking 
for access to the same object.  If a second process happens to decref 
'id' in between the first process creating the reference and 
incrementing it, then the incref from the first process will fail.  This 
is exactly what's happening (some of the time) in test_remote in the 
test_suite.  The failing sequence of operations is:

initial state:         id not in id_to_refcount
(process 1): create    id_to_refcount[id] is None
(process 1): incref    id_to_refcount[id] == 1
(process 2): create    id_to_refcount[id] == 1
(process 1): decref    id not in id_to_refcount
(process 2): incref    KeyError!
(process 2): decref    (never get here...)

I'm not really familiar enough with the whole multiprocessing module to 
know what the right fix for this is.

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3419>
_______________________________________