Python threading (was: Re: global interpreter lock not working as it should)

[Martin v. Loewis]

bokr at oz.net (Bengt Richter) writes:

> >> A mutex should result in a context switch every time there is a
> >> release with a waiter present, since the releaser would reliably
> >> fail to re-acquire.  Perhaps that is the way it works on BSD?
> >> That might at least partly explain Jonathan's results.

> >I doubt that. 
> Why?

Because the other threads waiting for the GIL do not block on a
mutex. So mutex wait lists should not be relevant for this behaviour.

> IOW, I don't think the new owner has to execute before it becomes owner
> of the mutex, I think the OS hands it over as part of the release operation
> if there is a waiter. Why would it be done otherwise?

Because of this code in thread_pthread.h

	status = pthread_mutex_lock( &thelock->mut );
	CHECK_STATUS("pthread_mutex_lock[1]");
	success = thelock->locked == 0;
	if (success) thelock->locked = 1;
	status = pthread_mutex_unlock( &thelock->mut );
	CHECK_STATUS("pthread_mutex_unlock[1]");

	if ( !success && waitflag ) {
		/* continue trying until we get the lock */

		/* mut must be locked by me -- part of the condition
		 * protocol */
		status = pthread_mutex_lock( &thelock->mut );
		CHECK_STATUS("pthread_mutex_lock[2]");
		while ( thelock->locked ) {
			status = pthread_cond_wait(&thelock->lock_released,
						   &thelock->mut);
			CHECK_STATUS("pthread_cond_wait");
		}
		thelock->locked = 1;
		status = pthread_mutex_unlock( &thelock->mut );
		CHECK_STATUS("pthread_mutex_unlock[2]");
		success = 1;
	}

Nobody is blocking on the mutex; the other thread blocks on the
condition variable instead. The first thread signals the condition
variable, then tries to lock the mutex. If that succeeds, it will
reacquire the lock. The other thread will come out of the cond_wait,
and find that the lock is still locked. It then will wait on the
condition variable again (probably adding itself to the end of the
condition's wait list).

> OTOH, if there is a variable associated with the mutex that is
> supposed to represent some state of the interpreter, and other
> threads are reading this without synchronizing, then I can see a
> possible (different) race.

The ->locked field of the lock is protected by a mutex, so there is no
danger of it getting inconsistent - it is just not clear who will lock
it.

> If a race condition is possible, I think the OS mutex implementation
> is not good or more likely there's a bug in its use and/or simulation.

The OS mutex implementation is not (directly) used. Only the cond_wait
implementation (indirectly) tries to acquire the mutex.

> Because mutex is not used after 2.2? But I thought 2.2 was the
> 'business' edition.  If there's a race condition there, should it
> not be looked into and fixed?

There is no bug in the sense that inconsistency could occur. There is
just no guarantee that locks are fair in Python.

Regards,
Martin