[Python-bugs-list] [ python-Bugs-768649 ] popen4 doesn't close filedescriptors when in Threads

[SourceForge.net]

Bugs item #768649, was opened at 2003-07-09 13:36
Message generated for change (Comment added) made by gaul
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=768649&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: martin doudoroff (mdoudoroff)
Assigned to: Nobody/Anonymous (nobody)
Summary: popen4 doesn't close filedescriptors when in Threads

Initial Comment:


Running the following code under Linux will result in a

crash on the 508th thread started. The error is



    OSError: [Errno 24] Too many open files



The nature of the bug seems to be that Python isn't

closing filedescriptors cleanly when running a thread.



---------------------------------------

import os

from threading import Thread



class Crash(Thread):

    def run(self):

        a = os.popen4('ls')

        b = a[1].read()



        # uncommenting these lines fixes the problem

        #     but this isn't really documented as far as

        #     we can tell...

        # a[0].close()

        # a[1].close()



for i in range(1000):

    t = Crash()

    t.start()



    while t.isAlive():

         pass



    print i

---------------------------------------



The same code without threads (Crash as a plain class)

doesn't crash, so the descriptor must be getting taken

care of when the run() method is exited.



import os



class Crash:

    def run(self):

        a = os.popen4('ls')

        b = a[1].read()



for i in range(1000):

    t = Crash()

    t.run()



    print i



----------------------------------------------------------------------

Comment By: Andrew Gaul (gaul)
Date: 2003-10-01 13:51

Message:
Logged In: YES 
user_id=139865

Duplicated with Python 2.3 on Red Hat 7.3 using

glibc-2.2.5-43.  Popen3.{poll,wait} are written under the

incorrect assumption that waitpid can monitor any process in

the same process group, when it only works for immediate

children.  _active.remove is never called, so Popen3 objects

are never destroyed and the associated file descriptors are

not returned to the operating system.



A general solution for Popen[34] is not obvious to me.  With

patch #816059, popen2.popen[234] plugs the _active leak,

which in turn returns the file descriptors to the operating

system when the file objects that popen2.popen[234] return

go out of scope.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-07-09 22:20

Message:
Logged In: YES 
user_id=33168

I can't duplicate this on Redhat 9.  What OS, what version

of glibc and what kernel are you using?  Does it always

crash on the 508th iteration?



I tested with both 2.2.3 and 2.3b2 from CVS without

problems.  I even used ulimit to set my open files to 10.



Can you try the patch in bug #761888 to see if that helps? 

http://pythong.org/sf/761888 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=768649&group_id=5470