Logging thread with Queue and multiple threads to log messages

[Vinay Sajip]

On Nov 9, 8:28 pm, "scriptlear... at gmail.com" <scriptlear... at gmail.com>
wrote:
> I am trying to put up a queue (through aloggingthread) so that all
> worker threads can ask it to log messages.  However, the problem I am
> facing is that, well, theloggingthread itself is running forever.
> It does not know when it should exit.  Any suggestion?  None of the
> worker threads knows for sure when theloggingthread should exit
> since they don't know what other worker threads are doing.  I also try
> to set a time limit for theloggingthread, but it does not work.
>
> # mylogging.py - used by everyone else that needs to log messages
> import threading
> import Queue
> import time
>
> # This is the queue object that's used to hold log events.
> # LoggingThread knows about it, and the log() function below
> # knows about it.  No one else is allowed to see it.
> _log_queue = Queue.Queue()
>
> class LoggingThread(threading.Thread):
>     def __init__(self,logfile,duration):
>         threading.Thread.__init__(self)
>         self.logfile = logfile
>         self.duration = duration
>
>     def run(self):
>         f=open(self.logfile, 'w')
>         #it's probably not the right way to set an end time here
>         #since theloggingthread should wait until all worker threads
> are done
>         end = time.time() + self.duration + 10
>         while(end > time.time()):
>             message = _log_queue.get()
>             f.write(message + '\n')  # (or whatever)
>
>         f.close()
>
> # Kick off theloggingthread.
> LoggingThread("logs/myapp.log",20).start()
>
> def log(msg):
>     _log_queue.put(msg)
>
> class Worker(threading.Thread):
>     def __init__(self,no,duration):
>         threading.Thread.__init__(self)
>         self.no = no
>         self.duration = duration
>
>     def run(self):
>         end = time.time() + self.duration
>
>         while(end > time.time()):
>             log('Thread Object (%d):(%d), Time:%s in seconds %d'%
> (self.no,self.duration,time.ctime(),time.time()))
>             time.sleep(10)
>
>         print 'Done with worker (%d) at %s'%(self.no,time.ctime())
>
> def main():
>     children = []
>     args = parseArgs()
>
>     for i in range(args.threads):
>         log('i=%d'%(i))
>         children.append(Worker(i,args.duration))
>         children[i].start()
>         time.sleep(0.1)

Take a look at this example test script to see how to use logging in a
multi-threaded environment:

http://dpaste.com/hold/89734/

Regards,

Vinay Sajip