global interpreter lock not working as it should

[Jonathan Hogg]

On 30/7/2002 21:43, in article
mailman.1028058230.6943.python-list at python.org, "brueckd at tbye.com"
<brueckd at tbye.com> wrote:

> [...] Sure the GIL makes sure that Python object internals are
> fiddled with one thread at a time, but if you have only 1 CPU, there's
> only one thread running at a time anyway, regardless of whether your
> program is in C or Python.

Err... I believe that's why I prefixed my comments with:

"If you have a multiple-CPU machine and a kernel-level thread implementation
and a multi-threaded CPU-intensive application, [...]."

>> Python threads just aren't any good for parallelism.
> 
> Remove the word 'Python' and your statement is more correct. Threads
> themselves aren't all that good for parallelism, which is fine because
> that's not what they're for anyway. [...]

No, I was quite careful in what I said. A good kernel-level threads
implementation is perfectly capable of exploiting parallelism on multiple
CPU machines. When I say Python threads I mean threads executing Python
code. The GIL requires that this happen with mutual exclusion.

>> They're mainly useful as a communication abstraction.
> 
> Sorry, but that's just plain wrong. Python threads are useful for the
> exact same things threads are useful for in Java, C, etc. (handling
> multiple blocking tasks simultaneously, doing "stuff" while waiting for
> user input, etc.).

Unless you have a bunch of threads sitting around in timewait for the sheer
heck of it, all of the blocking that threads do is on external I/O or
inter-thread communication. There's no program of this kind that couldn't
have been written using an event model instead (and probably more
efficiently). Hence my point: they are mainly useful as a communication
abstraction.

>> You can browse the archives of this group for many (many) discussions on
>> "fixing" the GIL, but you aren't going to get a solution.
> 
> That discussion typically revolves around multi-CPU systems, but the OP
> didn't mention having multiple CPUs.

If the OP isn't trying to exploit parallelism then why write multiple
CPU-bound threads? If his threads were blocking to do I/O he'd have no
problem, so clearly they are doing CPU intensive work. If there's more than
one of them he's trying to do CPU intensive work in parallel. If he doesn't
have a multiple CPU machine and a kernel-level threads implementation then
this will always be an exercise in futility.

All of which brings me back around to my original point:

"I think the main problem is that you're trying to do something with Python
threads that they just weren't designed for: CPU intensive work."

I really do think we're singing from the same hymn sheet here.

-no-you're-right!-ly y'rs,

Jonathan