how can a child thread notify a parent thread its status?

[Piet van Oostrum]

>>>>> MRAB <python at mrabarnett.plus.com> (M) wrote:

>M> scriptlearner at gmail.com wrote:
>>> I decided to go with one big log file, which will be shared by all
>>> threads (child and parent).  A log message Queue is used to store all
>>> log entries, and a customized logger thread will get log entries from
>>> the Queue.
>>> 
>>> #from the logger thread#
>>> def run(self):
>>> while self.flag == 1: #if the flag is set to 0, the logger
>>> thread should exit
>>> try:
>>> entry = self.q.get()
>>> except Empty:
>>> self.logger.debug('cant find any log entry')
>>> continue
>>> except:
>>> self.logger.error("Unexpected error:", sys.exc_info()
>>> [0])
>>> raise
>>> #do whatever that should be done
>>> self.logger.info("logger thread done") #should see this
>>> message in log file as well
>>> def off(self):
>>> self.logger.info('turning off flag')
>>> self.flag = 0
>>> 
>>> 
>>> #in parent thread#
>>> logItemQ.put('We are done, lets stop the logger now.')
>>> time.sleep(1) #it seems that the logger thread cannot exit if
>>> I put a sleep here
>>> myLog.off() #off is called successfully
>>> myLog.join()
>>> 
>>> 
>>> I put an off method to turn off a flag so the logger thread knows it
>>> should exit.  However, the last log message (the one 'We are done,
>>> lets stop the logger now.') won't be logged if I call myLog.off() and
>>> myLog.join() immediately.  So, I put a time.sleep(1) to make sure the
>>> logger thread has enough time to finish it's job.  Unfortunately, now
>>> the logger thread simply won't exit, and I don't see the message
>>> 'logger thread done'.  I can't figure out at which point it hangs,
>>> since I don't any new log entry but the thread simply won't exit.
>>> Am I taking a right approach by using a flag?  Should I lock the flag?

>M> self.q.get() will block if the queue is empty, so the Empty exception is
>M> never raised.

>M> What's happening is that the parent thread puts the final message into
>M> the queue, sleeps, and then clears the flag; meanwhile, the logging
>M> thread gets the message, writes it out, checks the flag, which is still
>M> set, and then tries to get the next message. The queue is empty, so the
>M> .get() blocks.

>M> The simplest solution is not to use a flag, but the sentinel trick. The
>M> parent thread can put, say, None into the queue after the last message;
>M> when the logging thread gets None, it knows it should terminate.

To the OP: a sleep will never do it because you can't be sure how long
the logging thread will lag behind. Most multithreaded `solutions' which
depend on timings are buggy.

If you have more than one thread writing to the log queue one sentinel
won't solve the problem. You would need as many sentinels as there are
threads, and the logger should count the sentinels which can have normal
messages between them. It may be a bit fragile to depend on the number
of threads especially when this number is dynamic. One solution would be
to have special StartLogging and StopLogging objects that are put in the
queue when a thread starts and finishes respectively. The logger can stop
then when the number of StopLogging objects equals the number of
StartLogging objects received and that number is greater than 0. This
presupposes that no new thread will be started after all the others have
finished. The StartLogging objects can also be put in the queue by the
parent when it starts up the threads, which may be more robust because
it has fewer timing dependencies.
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org