sleep and Timer

[Bengt Richter]

On Thu, 29 May 2003 15:39:52 +0800, "Rob Hall" <robhall at ii.net> wrote:

>
>> >def run(self):
>> >    while self.quit = false:
>> >        DO SOME STUFF
>> >        time.sleep(120)
>> What if instead of sleeping you had an event object and waited for it with
>a timeout of 120, e.g.,
>>          evt.wait(120)
>>          if evt.isSet(): # means another thread set the event, meaning
>kill
>>              # die here -- should be immediately after event is set
>>          #otherwise just let loop continue around
>> >
>>
>
>I rather like this approach.  It works well.
>
>The trouble is, once I go above 200 threads, it starts to bog down.
>
Well, I meant having all the threads wait on the *same* event, not one each.

>See my other post for an example using sleep(), together with some stats.
>
>With the same problem here, using your solution, I get the following stats:
>
>Time to create 500 threads:  216.85
>Time to kill all threads:    34.72
>
With a single event my result went from

Time taken to create threads is 176.814000.
Time taken to kill threads is 67.717000.

to

Time taken to create threads is 122.987000.
Time taken to kill threads is 1.582000.

and by separating creating from starting the threads, I get

Time taken to create threads is 0.251000.
Time taken to start threads is 120.763000.
Time taken to kill threads is 1.402000.


>As you can see, when working with a large number of threads, it isn't quite
>upt-to-speed (excuse the pun).
>
Well, the starting of the threads is progressively time sharing with the started threads,
so if we make them all wait for a run event immediately after they start, they should
get out of the way quickly and allow the starting to go faster. Then we can trigger them
all to continue by setting the single run event. The result is then (capturing a log to
a file apparently also speeded things a bit, avoiding screen scrolling ;-)

Thread 1 launched
Thread 2 launched
...
hread 499 launched
Thread 500 launched
Thread 1 running
Thread 2 running
...
hread 498 running
Thread 499 running
Thread 500 running
Exiting thread 1.
>>>> Thread 170 terminated.
>>>> Thread 409 terminated.
>>>> Thread 162 terminated.
... (out of order, but all accounted for ) ...
>>> Thread 500 terminated.
>>>> Thread 120 terminated.
>>>> Thread 76 terminated.
>>>> Thread 14 terminated.
Time taken to create threads is 0.260000.
Time taken to start threads is 0.471000.
Time taken to run-trigger threads is 3.465000.
Time taken to kill threads is 1.352000.

>But I love the solution for a small number of threads!  In this case it is
>comparible in speed.  Thanks!
>
I don't know the real requirements of your problem, but I don't see overwhelming
overhead, just normal diminishment of the main thread's share of CPU time. But if
you don't let them run until you fire the starting gun as a single event setting,
you don't get all that interference.

I made the events single class variables instead of individual instance variables,
so the code changes are minimal. Below is the last version experimented with.
At least some of the changes are flagged with #XXX# ;-)

====< RobHall.py >=======================================================
import time
import threading
import random

class test(threading.Thread):
    evt = threading.Event() #XXX#
    evt_run = threading.Event() #XXX#
    def __init__(self, number):
        threading.Thread.__init__(self)
        self.terminate = 0
        self.number = number
        #XXX# self.evt = threading.Event()

    def run(self):
        print 'Thread %d launched' % self.number
        self.evt_run.wait() #XXX#
        print 'Thread %d running' % self.number #XXX#
        sleepTime = random.random() * 10
        while 1:
            self.evt.wait(sleepTime)
            if self.evt.isSet():
                break

        print '>>>> Thread %d terminated.' % self.number


    def quit(self):
        print 'Exiting thread %d.' % self.number
        self.evt.set()


# Create the threads
startCreate = time.time()
threadList = []
for number in range(1,501):
    t = test(number)
    #XXX# t.start()
    threadList.append(t)
endCreate = time.time()
startStart = time.time()
for t in threadList:
    t.start()
endStart = time.time()
startRun = time.time()
threadList[0].evt_run.set()
endRun = time.time()

time.sleep(2)

# Kill the threads
startKill = time.time()
for t in threadList[:1]: #XXX# only need one call
    t.quit()
for t in threadList:
    t.join()

endKill = time.time()

# Display results
timeCreate = endCreate - startCreate
timeStart = endStart - startStart
timeRun = endRun - startRun
timeKill = endKill - startKill
print 'Time taken to create threads is %f.' % timeCreate
print 'Time taken to start threads is %f.' % timeStart
print 'Time taken to run-trigger threads is %f.' % timeRun
print 'Time taken to kill threads is %f.' % timeKill
=========================================================================
Regards,
Bengt Richter