threading priority
David Bolen
db3l at fitlinxx.com
Mon Dec 20 12:32:13 EST 2004
Peter Hansen <peter at engcorp.com> writes:
> alecwy at gmail.com wrote:
> > I googled as suggested, and the answer isn't crystal clear. My
> > impression is that the problem is that a python thread must acquire the
> > GIL in order to execute, and the strategy for deciding which thread
> > should get the GIL when multiple threads are waiting for it is not
> > based on priority. Is that correct?
>
> That's basically correct. I don't actually know what
> the strategy is, though I suspect it's either not
> formally documented or explicitly not defined, though
> for a given platform there may be some non-arbitrary
> pattern...
> (...)
I expect the Python interpreter has little to say over thread
prioritization and choice of execution, although it does impose some
granularity on the rate of switching. The GIL itself is implemented
on the lower layer lock implementation, which is taken from the native
threading implementation for the platform.
Therefore, when multiple Python threads are waiting for the GIL, which
one is going to get released will depend on when the underlying OS
satisfies the lock request from the threads, which should be based on
the OS thread scheduling system and have nothing to do with Python
per-se.
I do believe you are correct in that the Python GIL prevents thread
pre-emption by the OS (because all other Python threads are waiting on
the GIL and not in a running state), but the actual act of switching
threads at a switching point (sys.setcheckinterval()) would be an OS
only decision, and subject to whatever standard platform thread
scheduling rules were in place.
So if you were to use a platform specific method to control thread
priority, that method should be honored by the Python threads (subject
to the granularity of the system check interval for context switches).
For example, here's a Windows approach that fiddles with the thread
priority:
- - - - - - - - - - - - - - - - - - - - - - - - -
import threading
import ctypes
import time
w32 = ctypes.windll.kernel32
THREAD_SET_INFORMATION = 0x20
THREAD_PRIORITY_ABOVE_NORMAL = 1
class DummyThread(threading.Thread):
def __init__(self, begin, name, iterations):
threading.Thread.__init__(self)
self.begin = begin
self.tid = None
self.iterations = iterations
self.setName(name)
def setPriority(self, priority):
if not self.isAlive():
print 'Unable to set priority of stopped thread'
handle = w32.OpenThread(THREAD_SET_INFORMATION, False, self.tid)
result = w32.SetThreadPriority(handle, priority)
w32.CloseHandle(handle)
if not result:
print 'Failed to set priority of thread', w32.GetLastError()
def run(self):
self.tid = w32.GetCurrentThreadId()
name = self.getName()
self.begin.wait()
while self.iterations:
print name, 'running'
start = time.time()
while time.time() - start < 1:
pass
self.iterations -= 1
if __name__ == "__main__":
start = threading.Event()
normal = DummyThread(start, 'normal', 10)
high = DummyThread(start, 'high', 10)
normal.start()
high.start()
# XXX - This line adjusts priority - XXX
high.setPriority(THREAD_PRIORITY_ABOVE_NORMAL)
# Trigger thread execution
start.set()
- - - - - - - - - - - - - - - - - - - - - - - - -
And the results of running this with and without the setPriority call:
Without: With:
normal running high running
high running high running
normal running high running
high running high running
normal running normal running
high running high running
normal running high running
high running high running
normal running high running
high running normal running
normal running high running
high running high running
normal running normal running
high running normal running
normal running normal running
high running normal running
normal running normal running
high running normal running
normal running normal running
high running normal running
I'm not entirely positive why the normal thread gets occasionally
executed before the high thread is done. It might be that the
interpreter is actually releasing the GIL in the code I've written for
the thread's run() (maybe during the I/O) which opens up an
opportunity, or it may be that Windows is boosting the other thread
occasionally to avoid starvation. So I expect the normal thread is
getting occasional bursts of bytecode execution (the syscheckinterval).
But clearly the OS level prioritization is largely driving things.
-- David
More information about the Python-list
mailing list