Threading problem

[David Bolen]

"Larry Whitley" <ldw at us.ibm.com> writes:

> I have a PC with two processors at work and have decided to try to use
> threading to run some long running console applications.  My basic idea is
> to start two threads, one for each processor, and give them a list of things
> to do.  As each thread finishes it one job will go to the list and pick off
> the next thing to do.  When the list is empty, both threads are done.

Ignoring the specific coding error that has been addressed by other
responses, it should also be pointed out that if you're trying to get
effective utilization of your pair of CPUs, this probably won't do it.

The Python global interpreter lock requires that even with native
threading, only one thread is actually interpreting Python code at any
moment in time.  The net result of this is that while threading is
excellent for overlapping I/O, it won't efficiently distribute CPU
amongst processors.  That's because if you are CPU bound, then you are
stuck in Python code execution, and only one processor can be doing
that at any given moment in time.

This can be improved somewhat if you use extensions that release the
GIL, but in most cases that's done during I/O and not processing time,
particularly if the latter will require access to Python structures.
But in theory a purely CPU bound extension could release the GIL while
performing computations that it knew didn't require any interface to
Python.

--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/