The Future of Python Threading

[Nick Craig-Wood]

Bjoern Schliessmann <usenet-mail-0306.20.chr0n0ss at spamgourmet.com> wrote:
>  Justin T. wrote:
> 
> > The detrimental effects of the GIL have been discussed several
> > times and nobody has ever done anything about it. 
> 
>  Also it has been discussed that dropping the GIL concept requires
>  very fine locking mechanisms inside the interpreter to keep data
>  serialised. The overhead managing those fine ones properly is not
>  at all negligible, and I think it's more than managing GIL.

That is certainly true.  However the point being is that running on 2
CPUs at once at 95% efficiency is much better than running on only 1
at 99%...

> > The truth is that the future (and present reality) of almost every
> > form of computing is multi-core,
> 
>  Is it? 8)

Intel, AMD and Sun would have you believe that yes!

>  The question is: If it really was, how much of useful performance
>  gain would you get?

The linux kernel has been through these growing pains already...  SMP
support was initially done with the Big Kernel Lock (BKL) which is
exactly equivalent to the GIL.

The linux kernel has moved onwards to finer and finer grained locking.

I think one important point to learn from the linux experience is to
make the locking a compile time choice.  You can build a uniprocessor
linux kernel with all the locking taken out or you can build a
multi-processor kernel with fine grained locking.

I'd like to see a python build as it is at the moment and a python-mt
build which has the GIL broken down into a lock on each object.
python-mt would certainly be slower for non threaded tasks, but it
would certainly be quicker for threaded tasks on multiple CPU
computers.

The user could then choose which python to run.

This would of course make C extensions more complicated...

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick