[Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Eric Snow]

On Wed, Jul 18, 2018 at 1:37 AM Barry Scott <barry at barrys-emacs.org> wrote:
> Let me try a longer answer. The inc+test and dec+test do not require a
> lock if coded correctly. All OS and run times have solved this to provide
> locks. All processors provide the instructions that are the building blocks
> for lock primitives.
>
> You cannot mutate a mutable python object that is not protected with the GIL as
> the change of state involves multiple parts of the object changing.
>
> If you know that an object is immutable then you could only do a check on the
> ref count as you will never change the state of the object beyond its ref count.
> To access the object you only have to ensure it will not be deleted, which the
> ref count guarantees. The delete of the immutable object is then the only job
> that the original interpreter must do.

Perhaps we're agreeing?  Other than the single decref at when
"releasing" the object, it won't ever be directly modified (even the
refcount) in the other interpreter.  In effect that interpreter holds
a reference to the object which prevents GC in the "owning"
interpreter (the corresponding incref happened in that original
interpreter before the object was "shared").  The only issue is how to
"release" the object in the other interpreter so that the decref
happens in the "owning" interpreter.  As earlier noted, I'm planning
on taking advantage of the exiting ceval "pending calls" machinery.

So I'm not sure where an atomic int would factor in.  If you mean
switching the exiting refcount to an atomic int for the sake of the
cross-interpreter decref then that's not going to happen, as Ronald
suggested.  Larry could tell you about his Gilectomy experience. :)

Are you suggesting something like a second "cross-interpreter
refcount", which would be atomic, and add a check in Py_DECREF?  That
would imply an extra cross-interpreter-oriented C-API to parallel
Py_DECREF.  It would also mean either adding another field to PyObject
(yikes!) or keeping a separate table for tracking cross-interpreter
references.  I'm not sure any of that would be better than the
alternative I'm pursuing.  Then again, I've considered tracking which
interpreters hold a "reference" to an object, which isn't that
different.

-eric