[Python-Dev] Fwd: Removal of GIL through refcounting removal.

[Michael Foord]

Guido van Rossum wrote:
> On Thu, Oct 30, 2008 at 10:31 AM, Eric Smith <eric at trueblade.com> wrote:
>   
>> Guido van Rossum wrote:
>>     
>>> No offense taken. The V8 experience makes me feel much more optimistic
>>> that they might actually pull this off. (I'm still skeptical about
>>> support for extension modules, withougt which CPython is pretty lame.)
>>>       
>> The need to modify all extension modules is the usual non-starter I see
>> mentioned when this topic comes up. The OP really needs to think about that
>> issue.
>>     
>
> It's a non-starter for immediate world-domination. But if they get the
> core to be significantly faster I expect there will be motivation to
> port extensions. There's also the PyPy effort to replace extension
> modules with ctypes-based wrappers. I could also imagine that
> extensions could be run in a sandbox that *does* use the equivalent of
> the GIL.
>
>   
For IronClad [1], an implementation of the Python C API in C# to allow 
you to use Python C extensions from IronPython, we take a hybrid 
approach to garbage collection.

We hold a strong reference to objects created by extensions (preventing 
.NET from garbage collecting them) and allow the extension to manipulate 
the reference count in the usual way. Periodically we scan our "managed 
objects" [2] to see if there are objects whose reference count has 
dropped to zero and we then delete our strong reference - allowing 
garbage collection to happen normally.

We have also implemented a GIL for the C extensions even though 
IronPython has no GIL.

Simple extension modules can already be used with IronPython through 
Ironclad, and for Numpy (our goal) arrays can be created and used and we 
are working on 'everything else'.

Moving more C extensions to an implementation based on ctypes would be 
enormously useful for PyPy, IronPython and Jython, but ctypes is not yet 
as portable as Python itself which could be an issue (although one worth 
resolving).

Michael Foord


[1] http://code.google.com/p/ironclad
[2] Strictly speaking they are managed objects of unmanaged types, but 
we really need a better way of talking about them.

-- 
http://www.ironpythoninaction.com/