threads and sleep?

[Jonathan Ellis]

Jeffrey Maitland wrote:
> The problem I have is I had an application
> (wrote/co-wrote) that has a long run time dependant on some variables
> passed to it (mainly accuracy variables, the more accurate the longer
> the run time - makes sense). However in the hopes to speed it up I
> decided to write a threaded version  of the program to try and speed
> it up.  How ever what I am noticing is that the threaded version is
> taking as long possibly longer to run.  The thing is the threaded
> version is running on an 8 ia-64 proccessor system and it seems to
> only be using 2 or 3 porcessors at about 30% (fluxiates).  My guess is
> that 6 threads are running they are using 30% sprox each of a 2 given
> CPUS.

In many ways, Python is an incredibly bad choice for deeply
multithreaded applications.  One big problem is the global interpreter
lock; no matter how many CPUs you have, only one will run python code
at a time.  (Many people who don't run on multiple CPUs anyway try to
wave this off as a non-problem, or at least worth the tradeoff in terms
of a simpler C API, but with multicore processors coming from every
direction I think the "let's pretend we don't have a problem" approach
may not work much longer.)

If the GIL isn't an issue (and in your case it clearly is), you'll
quickly find that there's little support for debugging multithreaded
applications, and even less for profiling.

Sometimes running multiple processes is an acceptable workaround; if
not, good luck with the rewrite in Java or something else with real
thread support.  (IIRC Jython doesn't have a GIL; that might be an
option too.)

Python is a great tool but if you really need good threading support
you will have to look elsewhere.

-Jonathan