Python threading (was: Re: global interpreter lock not working as it should)

[Armin Steinhoff]

Jonathan Hogg <jonathan at onegoodidea.com> wrote in message news:<B97159DB.F0A9%jonathan at onegoodidea.com>...
> On 2/8/2002 20:38, in article
> ddc19db7.0208021138.5e3d1a61 at posting.google.com, "Armin Steinhoff"
> <a-steinhoff at web.de> wrote:
> 
> > If you run the Python interpreter with FIFO or RR scheduling ... the
> > following code from ceval.c make no sense, because the priority of the
> > interpreter is static.
> 
> Now I'm confused as to what you mean. The interpreter in Python is not a
> thread and thus has no priority of any kind.

When you start the program 'python' (RR/FIFO sched) then you are
creating a process (thread container) running with a static (fixed)
priority. This prioriy will be inherited by the python threads.

 
> The interpreter in Python is
> effectively a critical region of code. Sections of the interpreter that
> might block are placed outside of the region so that other threads can enter
> it while that thread is blocked. Similarly, every thread is forced
> periodically to leave the region and re-enter it in order to allow the
> thread scheduler to re-schedule as necessary (the piece you posted).

IMHO ... there is nothing like a 'thread-scheduler' in the python
code.
All python threads are scheduled by the OS ... that means there is no
code in the python program which can _force_ periodically a thread to
leave this critical section ( e.g. in the middle of the execution of
the 10 byte codes).
 
> With FIFO or RR scheduling, and assuming no blocking I/O, all threads will
> be available to run when the GIL is released. If the thread previously
> running before the GIL was released still has timeslice left it will be
> allowed to continue. If it has run out of timeslice then the next thread in
> line will be switched to. It will acquire the GIL and begin executing.

This happens only if the thread exhausting its timeslice before it
acquires again the GIL.

In all other cases the thread will be supended after exhausting its
timeslice (RR sched) and will own the GIL further .. other tasks will
be started and immediatly supended when they are trying to get the
GIL.
Have in mind that _ALL_ threads are running at the same
(ficed)priority level.
  
> There is nothing wrong with the code in ceval.c in this regard.

Releasing the GIL can't create a context switch if all threads running
at the same priority ... that's the problem.
 
> > That means, if a Python thread with a higher priority acquieres the
> > GIL, the running thread with the lower priority will inherit the
> > priority of the other thread (if priority inheritance is supported, if
> > not you will have the case of priority inversion ...) in order to
> > complete its task as soon as possible.
> > In such a case we will see a context switch between the realease_lock
> > and the acquire_lock calls.
> 
> Priority inversion is actually extremely unlikely in Python because the main
> shared resources is the GIL. There is only one of them so all threads that
> are not waiting on I/O require it. Therefore a medium priority thread will
> be unable to pre-empt the lower priority thread (or more accurately it will
> pre-empt the lower priority thread, immediately attempt to obtain the GIL,
> and block allowing the lower-priority thread which holds the GIL to
> continue) until the GIL is released, at which point the highest priority
> thread will be scheduled.

Priority inversion happens if a lower priority thread is blocking a
higher priority thread ... IMHO.

> 
> > Hope it's now clear how strong the thread behavior depends on the OS
> > and the used scheduling startegies.
> 
> But that's pretty much what everyone was trying to say all along. Python's
> threads are just like any other thread on the system and rely on the native
> thread scheduler to do what it thinks is best. Because of this, there is
> largely nothing that Python can (or indeed should) do to affect this
> scheduling.

Yes ... and that's the reason why different scheduling strategies have
influence on the scheduling of 'python threads'.

Ok, I will do some tests with the System Analyse Tool (SAT) of QNX6 PE
... just to see what happens in detail when I use the different
scheduling strategies.

Cheers

Armin 

> 
> Jonathan