Python threading (was: Re: global interpreter lock not working as it should)

[Jonathan Hogg]

On 2/8/2002 20:38, in article
ddc19db7.0208021138.5e3d1a61 at posting.google.com, "Armin Steinhoff"
<a-steinhoff at web.de> wrote:

> If you run the Python interpreter with FIFO or RR scheduling ... the
> following code from ceval.c make no sense, because the priority of the
> interpreter is static.

Now I'm confused as to what you mean. The interpreter in Python is not a
thread and thus has no priority of any kind. The interpreter in Python is
effectively a critical region of code. Sections of the interpreter that
might block are placed outside of the region so that other threads can enter
it while that thread is blocked. Similarly, every thread is forced
periodically to leave the region and re-enter it in order to allow the
thread scheduler to re-schedule as necessary (the piece you posted).

With FIFO or RR scheduling, and assuming no blocking I/O, all threads will
be available to run when the GIL is released. If the thread previously
running before the GIL was released still has timeslice left it will be
allowed to continue. If it has run out of timeslice then the next thread in
line will be switched to. It will acquire the GIL and begin executing.

There is nothing wrong with the code in ceval.c in this regard.

> That means, if a Python thread with a higher priority acquieres the
> GIL, the running thread with the lower priority will inherit the
> priority of the other thread (if priority inheritance is supported, if
> not you will have the case of priority inversion ...) in order to
> complete its task as soon as possible.
> In such a case we will see a context switch between the realease_lock
> and the acquire_lock calls.

Priority inversion is actually extremely unlikely in Python because the main
shared resources is the GIL. There is only one of them so all threads that
are not waiting on I/O require it. Therefore a medium priority thread will
be unable to pre-empt the lower priority thread (or more accurately it will
pre-empt the lower priority thread, immediately attempt to obtain the GIL,
and block allowing the lower-priority thread which holds the GIL to
continue) until the GIL is released, at which point the highest priority
thread will be scheduled.

> Hope it's now clear how strong the thread behavior depends on the OS
> and the used scheduling startegies.

But that's pretty much what everyone was trying to say all along. Python's
threads are just like any other thread on the system and rely on the native
thread scheduler to do what it thinks is best. Because of this, there is
largely nothing that Python can (or indeed should) do to affect this
scheduling.

Jonathan