Python threading (was: Re: global interpreter lock not working as it should)
Armin Steinhoff
a-steinhoff at web.de
Mon Aug 5 11:43:38 EDT 2002
Jonathan Hogg <jonathan at onegoodidea.com> wrote in message news:<B9715946.F0A5%jonathan at onegoodidea.com>...
> On 3/8/2002 2:43, in article aifcfh$jha$0 at 216.39.172.122, "Bengt Richter"
> <bokr at oz.net> wrote:
>
>
> I also tried the same code on a FreeBSD, Solaris, and Linux box. The summary
> results were:
>
> OS Architecture Python Thread Switches (avg 2 runs)
> Mac OS X 10.1 PPC 2.3-CVS 24971
> FreeBSD 4.3 i386 2.2.1 158499
> Linux 2.2.14 i386 2.2 18281
> Solaris 8 SPARC 2.2.1 243
>
>
Hi All,
I have build three versions of python by inserting a sched_yield and a
delay of
1ms in the code of ceval.c below ... and did run Jonathans testcode.
>From cevel.c ...
#ifdef WITH_THREAD
if (interpreter_lock) {
/* Give another thread a chance */
if (PyThreadState_Swap(NULL) != tstate)
Py_FatalError("ceval: tstate mix-up");
PyThread_release_lock(interpreter_lock);
/* Other threads may run now */ case: unmodified
sched_yield() case: sched_yield
delay(1) case: delay
PyThread_acquire_lock(interpreter_lock, 1);
if (PyThreadState_Swap(tstate) != NULL)
Py_FatalError("ceval: orphan tstate");
}
#endif
Here are the results:
case ceval.c unmodified:
>>> execfile('/root/threads.py')
Counts:
[207251, 189529, 228940, 203701, 216320, 169515, 218877, 223871,
185256, 212550]
Total = 2055810
Switches:
[85, 85, 83, 84, 84, 84, 84, 83, 84, 86]
Total = 842
>>>
That means the timeslice exhausted 85 times (or jumped to a higher
priority) at 'other threads may run now' during 207251 loops! (More
often than I could imagine ... )
This leads to an awful bad thread switching performance!
case sched_yield:
Counts:
[110635, 110589, 110598, 110597, 110617, 110585, 110600, 110597,
110604, 110587]
Total = 1106009
Switches:
[110584, 110549, 110559, 110558, 110589, 110549, 110561, 110553,
110562, 110548]
Total = 1105612
>>>
This says it all ....
case delay:
Counts:
[1438, 1434, 1431, 1428, 1425, 1422, 1419, 1416, 1412, 1409]
Total = 14234
Switches:
[1434, 1434, 1431, 1428, 1425, 1422, 1419, 1416, 1412, 1409]
Total = 14230
We have a slow interpreter ... but a good thread switching performance
:-)
-------------------------------------------------------------------------------
My conclusion: insert a sched_yield if the ceval.c code is called from
a thread.
#ifdef WITH_THREAD
if (interpreter_lock) {
/* Give another thread a chance */
if (PyThreadState_Swap(NULL) != tstate)
Py_FatalError("ceval: tstate mix-up");
PyThread_release_lock(interpreter_lock);
/* Other threads may run now */
--> if (called_from_a_thread)
--> sched_yield()
PyThread_acquire_lock(interpreter_lock, 1);
if (PyThreadState_Swap(tstate) != NULL)
Py_FatalError("ceval: orphan tstate");
}
#endif
Armin
More information about the Python-list
mailing list