Python threading (was: Re: global interpreter lock not working as it should)

[Bengt Richter]

On 06 Aug 2002 10:44:35 +0200, martin at v.loewis.de (Martin v. Loewis) wrote:

>bokr at oz.net (Bengt Richter) writes:
>
>> >POSIX specifies that pthread_cond_wait can return spontaneously, which
>> When my computer acts "spontaneous" I worry ;-) What kinds of spontaneity
>> are anticipated?
>
>I think no rationale is given; it probably simplifies the
>implementation on some systems. For example, it could be that threads
>get a SIGUSR1, which happens to be used as a wakeup signal in an
>implementation. People are usually at a loss explaining this aspect of
>POSIX - yet it is documented everywhere.
>
>> Yes, some kind of reference implementation to look at. I thought Linux
>> would be a good candidate. I guess it's expected to be part of glibc2
>> though, so it would just be extracted for a reference.
>
>This is the first time that this subject came up (atleast at that
>level of detail, atleast AFAIR); anybody who wants to research the
>details should know where to search. They should also know that
>linuxthreads most likely is *not* a typical representative of these
>things, since Linus refuses to support threads in the OS kernel (as
>opposed to merely supporting processes); until very recently.
>
Ok, I'm being lazy ;-) Do you mean that LinuxThreads is probably
not underneath Python threading in RH7.3, for example (which I will
install in a few weeks)?  I guess I could chase down make files and
dependencies etc. and determine what's linked into what system, but
as said, I'm being lazy ;-/

>> BTW, is there a Python module that can unpack rpms?
>
>I think the standard rpm distribution comes with Python modules, since
>Redhat does much of their RPM management in Python.
>
I have the RH7.3 CDs, but haven't installed, so I was stuck trying to
browse the sources which are apparently all rpm. So I have a chicken-egg
situation until I install the chicken. I had to go to their web CVS.

>> Well, I come back to the question of whether and/or when we want to enforce
>> handover of the lock, or otherwise modify the way it's held. 
>
>I don't think so: there is no problem to solve here. If every 10 byte
My impression is that some apparently think there is a latency problem
for some applications.

>code instructions there was a thread switch, you'd get quite a
>performance desaster.
Sure, but you wouldn't always do that. E.g., if you were monitoring another
condition than count-down-from-10 (or n), e.g., how long a _time_ you had
held the GIL, and whether higher priority waiters were waiting, then you could
hand over the GIL instead of throwing it up for grabs and yourself immediately grabbing.

In fact, if you were monitoring those things, you might not have to have the
overhead of releasing and acquiring blindly every 10 or n byte instructions.
Monitoring aforesaid things wouldn't necessarily have to be an expensive
polling operation, if the condition polled were externally set when it happened
instead of being determined in the loop.

>
>> It would be nice to know what the design intent for the optimum was,
>> even if that can't always be achieved.
>
>The intent (as I earlier said) is to
I saw it, thanks. Sorry. Things went out in mixed order.

>a) to operate similarly on a wide range of systems,
>b) minimise the overhead that threading takes, compared to a non-threaded
>   application, and
Seems like if unneeded calls to release the GIL could be eliminated cheaply,
that would be on target for this item?
>c) provide a minimum amount of fairness, if the underlying system has
>   some guarantee of fairness. IOW, starvation should not be observable
>   in real life.
Seems like if calls to release the GIL _and_ transfer ownership of the lock
could be implemented cheaply -- to be used _when appropriate_ to minimize context
switch latency -- that would be on target for this item?

Regards,
Bengt Richter