Kill GIL (was Re: multi threading in multi processor (computer))

[Jack Diederich]

On Sat, Feb 12, 2005 at 07:13:17PM -0500, Aahz wrote:
> In article <7xr7jlwieq.fsf at ruckus.brouhaha.com>,
> Paul Rubin  <http://phr.cx@NOSPAM.invalid> wrote:
> >
> >The day is coming when even cheap computers have multiple cpu's.
> >See hyperthreading and the coming multi-core P4's, and the finally
> >announced Cell processor.

I'm looking forward to Multi-core P4s (and Opterons).  The Cell is a
non-starter for general purpose computing.  Arstechnica has a couple
good pieces on it, the upshot is that it is one normal processor
with eight strange floating point co-processors hanging off it.

> >Conclusion: the GIL must die.
> 
> It's not clear to what extent these processors will perform well with
> shared memory space.  One of the things I remember most about Bruce
> Eckel's discussions of Java and threading is just how broken Java's
> threading model is in certain respects when it comes to CPU caches
> failing to maintain cache coherency.  It's always going to be true that
> getting fully scaled performance will require more CPUs with non-shared
> memory -- that's going to mean IPC with multiple processes instead of
> threads.
> 
> Don't get me wrong -- I'm probably one of the bigger boosters of threads.
> But it bugs me when people think that getting rid of the GIL will be the
> Holy Grail of Python performance.  No way.  No how.  No time.

"Me Too!"  for a small number of processors (four) it is easier (and
usually sufficient) to pipeline functional parts into different
processes than it is to thread the whole monkey.  As a bonus this usually
gives you scaling across machines (and not just CPUs) for cheap.  I'm 
aware there are some problems where this isn't true.  From reading this
thread every couple months on c.l.py for the last few years it is my 
opinion that the number of people who think threading is the only solution
to their problem greatly outnumber the number of people who actually have 
such a problem (like, nearly all of them).

Killing the GIL is proposing a silver bullet where there is no werewolf-ly,

-Jack