dual processor

[Nick Craig-Wood]

Jeremy Jones <zanesdad at bellsouth.net> wrote:
>  One Python process will only saturate one CPU (at a time) because
>  of the GIL (global interpreter lock).

I'm hoping python won't always be like this.

If you look at another well known open source program (the Linux
kernel) you'll see the progression I'm hoping for.  At the moment
Python is at the Linux 2.0 level.  Its supports multiple processors,
but has a single lock (Python == Global Interpreter Lock, Linux == Big
Kernel Lock).

Linux then took the path of splitting the BKL into smaller and smaller
locks, increasing the scalability over multiple processors.
Eventually by 2.6 we now have a fully preempt-able kernel, lock-less
read-copy-update etc.

Splitting the GIL introduces performance and memory penalties.  Its
been tried before in python (I can't find the link at the moment -
sorry!).  Exactly the same complaint was heard when Linux started
splitting its BKL.

However its crystal clear now the future is SMP.  Modern chips seem to
have hit the GHz barrier, and now the easy meat for the processor
designers is to multiply silicon and make multiple thread / core
processors all in a single chip.

So, I believe Python has got to address the GIL, and soon.

A possible compromise (also used by Linux) would be to have two python
binaries.  One with the BKL which will be faster on uniprocessor
machines, and one with a system of fine grained locking for
multiprocessor machines.  This would be selected at compile time using
C Macro magic.

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick