GIL in the new glossary

[Dave Brueck]

On Thursday 02 October 2003 01:20 pm, Luis P Caamano wrote:
> > The global interpreter lock can explicitly be released from C code.  Lots
> > of I/O code does this already.  Only one thread can be executing Python
> > byte code at a time, but multiple threads can execute non-interpreter C
> > code. (I'm sure someone more knowledgeable about threads will correct me
> > if I'm off-base here.)
>
> Although I understand this is possible and how it works,
> it still doesn't help me at all if all I write is
> python code.  My multi-threaded program works great, I take
> it to a big HP SuperDome with lots of processors and it just
> doesn't scale at all.  I have to go back to the drawing
> board and start playing around the problem.

It really depends on your application though - an I/O intensive app, for 
example, has good chances of scaling decently well without you having to do 
anything extra, as does any app that makes heavy use of a C extension that 
releases the GIL for its work. For many applications, a *lot* of time is 
spent in C libraries that have released the GIL.

Obviously this doesn't solve the problem for all applications, but to say it 
"doesn't help you at all" isn't necessarily true.

> Let me put it this way ... if python supported free
> threads and parallelism on multi-processor machines,
> I don't think I'd ever have to write a user space
> application in any other language, ever.  As it is right
> now, if I need to run a server app on a big MP machine
> and I need to scale accordingly, I have two choices:

It depends on what you mean by "server app", but if it's a network server (web 
server, file server, etc.) then that's actually a great example of when you 
_can_ get a decent degree MP scalability with the GIL. On a two-proc Linux 
box here in our office we have a Python server app that gets over 180% CPU 
utilization, so for now we haven't seen any reason to recode anything in C - 
performance is great. 

-Dave