Threading problem

[Cliff Wells]

On 03 Apr 2002 16:05:44 -0500
David Bolen wrote:
 
> Ignoring the specific coding error that has been addressed by other
> responses, it should also be pointed out that if you're trying to get
> effective utilization of your pair of CPUs, this probably won't do it.
> 
> The Python global interpreter lock requires that even with native
> threading, only one thread is actually interpreting Python code at any
> moment in time.  The net result of this is that while threading is
> excellent for overlapping I/O, it won't efficiently distribute CPU
> amongst processors.  That's because if you are CPU bound, then you are
> stuck in Python code execution, and only one processor can be doing
> that at any given moment in time.
> 
> This can be improved somewhat if you use extensions that release the
> GIL, but in most cases that's done during I/O and not processing time,
> particularly if the latter will require access to Python structures.
> But in theory a purely CPU bound extension could release the GIL while
> performing computations that it knew didn't require any interface to
> Python.

Ah.  That's pretty much what I thought.  Of course the solution would be to
write the system as two or more separate Python programs (perhaps started
by a single process) that use some form of IPC.  There's the added
complexity/overhead of the IPC mechanism, but you'd get full advantage of
SMP.

-- 
Cliff Wells, Software Engineer
Logiplex Corporation (www.logiplex.net)
(503) 978-6726 x308  (800) 735-0555 x308