[concurrency] Talk worth watching

[Guido van Rossum]

On Sun, Feb 19, 2012 at 8:57 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> As Dave's post and comments make clear, a lot of the problem with
>> switching Python's memory management comes from interfacing with external
>> libraries.  As long as one of Python's primary goals is to make it easy
>> to hook up external libraries, it will be difficult to get enough energy
>> toward removing the GIL.
>
> That's not necessarily a big problem. The ob_refcnt field could be kept
> for a certain duration as an "external references counter" that would
> prevent any disposal by the GC, until all 3rd party libraries have moved
> to hypothetical new APIs.
>
> In other words, reference counting doesn't have to be totally dropped,
> it has to be removed from the interpreter's critical paths (such as the
> eval loop, and many operations of builtin types).

This suggestion might actually show the way to gradually deprecating
the GIL and reference counts -- they can be kept as a crutch for C
code using the traditional Python/C API, while the core of interpreter
itself can move on to a different approach. Right now we're in the
situation where the core interpreter itself uses the traditional API,
but it wouldn't be impossible to switch to a new API while still
supporting the old one. Of course, there are lots of details to be
figured out, like what should happen at the boundaries. IIUC PyPy uses
a limited version of this approach (specifically with the purpose of
supporting extension modules) but it uses incompatible representations
for two two APIs, so crossing the border is a lot of work.

-- 
--Guido van Rossum (python.org/~guido)