[Python-Dev] GIL removal question

[Maciej Fijalkowski]

On Wed, Aug 10, 2011 at 1:43 PM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Aug 10, 2011 at 7:32 AM, David Beazley <dave at dabeaz.com> wrote:
>>
>> On Aug 10, 2011, at 6:15 AM, Nick Coghlan wrote:
>>
>>> On Wed, Aug 10, 2011 at 9:09 PM, David Beazley <dave at dabeaz.com> wrote:
>>>> You're forgetting step 5.
>>>>
>>>> 5. Put fine-grain locks around all reference counting operations (or rewrite all of Python's memory management and garbage collection from scratch).
>>> ...
>>>> After implementing the aforementioned step 5, you will find that the performance of everything, including the threaded code, will be quite a bit worse.  Frankly, this is probably the most significant obstacle to have any kind of GIL-less Python with reasonable performance.
>>>
>>> PyPy would actually make a significantly better basis for this kind of
>>> experimentation, since they *don't* use reference counting for their
>>> memory management.
>>>
>>
>> That's an experiment that would pretty interesting.  I think the real question would boil down to what *else* do they have to lock to make everything work.   Reference counting is a huge bottleneck for CPython to be sure, but it's definitely not the only issue that has to be addressed in making a free-threaded Python.
>
> They have a specific plan, based on Software Transactional Memory:
> http://morepypy.blogspot.com/2011/06/global-interpreter-lock-or-how-to-kill.html
>
> Personally, I'm not holding my breath, because STM in other areas has
> so far captured many imaginations without bringing practical results
> (I keep hearing about it as this promising theory that needs more work
> to implement, sort-of like String Theory in theoretical physics).

Note that the PyPy's plan does *not* assume the end result will be
comparable in the single-threaded case. The goal is to be able to
compile two *different* pypy's, one fast single-threaded, one
gil-less, but with a significant overhead. The trick is to get this
working in a way that does not increase maintenance burden. It's also
research, so among other things it might not work.

Cheers,
fijal