[issue9634] Add timeout parameter to Queue.join()

[Swen Wenzel]

Swen Wenzel added the comment:

I have another use case.
The Docs use the producer-consumer pattern as a usage example.
I'm also using this pattern but apparently the consumers are not that stable during development phase.

So if one of your consumers dies during execution of its task, before it can say 'task done', then you will have a deadlock.
You could of course avoid this by adding a finally block which will claim the task is done even if there was some error or exception but then you will lie to the producer!
And suppose you only have one consumer and there are still tasks waiting (which is the case for my application), then you'll still have a deadlock since nobody is there to execute the remaining tasks.
This could be solved by putting the exception handling within the consumer's mainloop like this:

Consumer(Thread):
    def __init__(self, queue):
        self.queue = queue

    def run():
        while True:
            task = self.queue.get()
            try:
                # execute task
            except:
                # handle exception (hopefully) without reraising one
            finally:
                self.queue.task_done()

This way, however, the producer won't notice any issues unless the consumer's exception handler sets a flag or puts the exception into a collection that can be checked by the producer.

But even that is no solution if the consumer executes a task with an endless loop or runs into a deadlock itself.

I would propose to throw an exception if queue.Queue.join() returns because of a timeout since then you can investigate the cause within the exception handler and do not have to check for the consumer's status after each join(). But this is microoptimization so I would also be satisfied with the same solution as for threading.Thread.join().

----------
nosy: +Swen.Wenzel

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9634>
_______________________________________