[Python-Dev] PEP 554 v3 (new interpreters module)

[Eric Snow]

On Tue, Oct 3, 2017 at 11:36 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> The problem relates to the fact that there aren't any memory barriers
> around CPython's INCREF operations (they're implemented as an ordinary
> C post-increment operation), so you can get the following scenario:
>
> * thread on CPU A has the sole reference (ob_refcnt=1)
> * thread on CPU B acquires a new reference, but hasn't pushed the
> updated ob_refcnt value back to the shared memory cache yet
> * original thread on CPU A drops its reference, *thinks* the refcnt is
> now zero, and deletes the object
> * bad things now happen in CPU B as the thread running there tries to
> use a deleted object :)

I'm not clear on where we'd run into this problem with channels.
Mirroring your scenario:

* interpreter A (in thread on CPU A) INCREFs the object (the GIL is still held)
* interp A sends the object to the channel
* interp B (in thread on CPU B) receives the object from the channel
* the new reference is held until interp B DECREFs the object

>From what I see, at no point do we get a refcount of 0, such that
there would be a race on the object being deleted.

The only problem I'm aware of (it dawned on me last night), is in the
case that the interpreter that created the object gets deleted before
the object does.  In that case we can't pass the deletion back to the
original interpreter.  (I don't think this problem is necessarily
exclusive to the solution I've proposed for Bytes.)

-eric