fd leak in parent process for logger.

[Roy Smith]

In article <e0d265f9-801d-4ab2-8c82-0d9723603435 at googlegroups.com>,
 ravindrapai34 at gmail.com wrote:

> Hi All,
>        I have server process which spawns a process for each request. Where 
>        parent process is leaking fd for logger. Please find example code.

You've got a lot of code here.  The first thing to do when trying to 
debug a problem like this is get rid of anything that's not essential to 
demonstrating the problem.  You import Thread, but don't appear to use 
it.  Get rid of the import line.  You also do this in a loop.  Can you 
demonstrate the same problem with a single pass?

Anyway, my guess here is that the logging package is keeping a reference 
to your log file when you create each FileHandler.  Try configuring the 
logger in your run() method instead of __init__().  I believe (untested) 
that will make the file get opened in the child process.


> 
> from threading import Thread
> from multiprocessing import Process
> from time import sleep
> import logging
> from uuid import uuid4
> 
> class ChildFile(object):
>     def __init__(self):
>         self.logger = logging.getLogger('test')
>         fh = logging.FileHandler('/tmp/test'+str(uuid4()))
>         fh.setLevel(logging.INFO)
>         self.logger.addHandler(fh)
>         self.fd = open('test2', 'wb')
> 
>     def run(self):
>         self.logger.info('dummy run')
>  
> def child_file_creator():
>     a = ChildFile()
>     child_process = Process(target=a.run)
>     child_process.start()
>     child_process.join()
> 
> if __name__ == '__main__':
>     print 'parent process run'
>     while True:
>         child_file_creator()
>         sleep(10)
> 
> 1) after child process exits.
> 2) For parent process, still fd remains open.
> one can check out using,
> 
> cd /proc/23223/fd
> 
> ravindra at ravindra-Ideapad-Z570:/proc/23223/fd$ ls -ltr
> total 0
> l-wx------ 1 ravindra ravindra 64 Nov 11 15:10 6 -> 
> /tmp/test62bba7f1-223c-4c17-a483-f6d92ab67222
> l-wx------ 1 ravindra ravindra 64 Nov 11 15:10 5 -> 
> /tmp/test2946cdf6-7e4c-4979-b56a-fd2cc6333398
> l-wx------ 1 ravindra ravindra 64 Nov 11 15:10 4 -> 
> /tmp/test0488579b-10d7-4635-abb0-a31a0ea79eeb
> lr-x------ 1 ravindra ravindra 64 Nov 11 15:10 3 -> /dev/urandom
> lrwx------ 1 ravindra ravindra 64 Nov 11 15:10 2 -> /dev/pts/19
> lrwx------ 1 ravindra ravindra 64 Nov 11 15:10 1 -> /dev/pts/19
> lrwx------ 1 ravindra ravindra 64 Nov 11 15:10 0 -> /dev/pts/19
> 
> 3) while normal file descriptor of open 'test2' is closed. But fd attached to 
> logger is leaking.
> 
> How can I close for the same for logger object.
> 
> Thanks and Regards,
> Ravindra M