CPython thread starvation

[John Nagle]

     I have a multi-threaded CPython program, which has up to four
threads.  One thread is simply a wait loop monitoring the other
three and waiting for them to finish, so it can give them more
work to do.  When the work threads, which read web pages and
then parse them, are compute-bound, I've had the monitoring thread
starved of CPU time for as long as 120 seconds.
It's sleeping for 0.5 seconds, then checking on the other threads
and for new work do to, so the work thread isn't using much
compute time.

    I know that the CPython thread dispatcher sucks, but I didn't
realize it sucked that bad.  Is there a preference for running
threads at the head of the list (like UNIX, circa 1979) or
something like that?

    (And yes, I know about "multiprocessing".  These threads are already
in one of several service processes.  I don't want to launch even more
copies of the Python interpreter.  The threads are usually I/O bound,
but when they hit unusually long web pages, they go compute-bound
during parsing.)

    Setting "sys.setcheckinterval" from the default to 10000 seems
to have little effect.  This is on Windows 7.

				John Nagle