[Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython)

[Nick Coghlan]

On Tue, Jul 27, 2010 at 12:33 AM, Ronald Oussoren
<ronaldoussoren at mac.com> wrote:
> In my opinion the GIL is a weak point of CPython and it would be nice if it
> could be fixed.  That is however easier said than done, a number of people
> have tried in the past and ran into implementation limitations like our
> refcounting garbage collector that make hard to remove the GIL without
> either rewriting lots of code, or running into a brick wall
> performance-wise.
>
> The HotPy presentation at EuroPython shows that it is possible to remove the
> GIL, although at the cost of replacing the garbage collector and most likely
> breaking existing C extensions (although the HotPy author seemed to have a
> possible workaround for that).

This is the kind of approach that seems to hold the most promise of
removing the GIL without incurring the single-threaded performance hit
that has been the achilles heel of previous attempts at creating a
free-threaded CPython implementation. With first IronClad and now PyPy
blazing the trail in interfacing a garbage collected Python
implementation with deterministic refcounting for C extension modules,
it seems plausible that this kind of approach may eventually prove
acceptable.

Furthermore, the with statement now provides a superior alternative to
application level tricks that previously relied on deterministic
refcounting.

While multi-threading does break down beyond a certain number of
cores, it *is* possible to do safely (particularly using queues to
pass references around) and can avoid plenty of serialisation overhead
when dealing with sizable data structures.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia