TaskQueue

[Rene Pijlman]

Raymond Hettinger:
>There are some competing approaches.  One is to attach sentinel objects
>to the end of the line as a way for consumer threads to know that they
>should shut down.  Then a regular t.join() can be used to block until
>the consumers threads have shut-down.  This approach is
>straight-forward, but it depends on 1) complicating the consumer logic
>to include sentinel detection and thread shut-down,

Writing this observation was way more complicated than writing the code
that's required to implement it :-)

    if task is None:
        break

I use 'None' as the sentinel. This is a larger snippet from my own
threadpool with the context of this code:

    def run(self):
        while True:
            task = self.workQueue.get()
            if task is None:
                break
            try:
                task.do()
            except:
                task.unhandledException()
            self.resultQueue.put(task)

>2) complicating the producer logic to append one sentinel for each consumer 
>when the data stream is done

        for i in range(self.numberOfThreads):
            self.workQueue.put(None)

Again, more characters in your observation than in the code.

>3) actually knowing when the data stream is done.  

    def doingWork(self):
        return self.numberOfTasks > 0

Which is possible because of:

    def putTask(self,task):
        self.workQueue.put(task)
        self.numberOfTasks += 1

    def getTask(self):
        task = self.resultQueue.get()
        self.numberOfTasks -= 1
        return task

-- 
René Pijlman

Wat wil jij leren?  http://www.leren.nl