[New-bugs-announce] [issue27448] Race condition in subprocess.Popen which causes a huge memory leak

[Andrew]

New submission from Andrew:

We had problem where at some point python start consuming RAM. Until it ends.

The reason was race condition in subprocess.Popen. Which triggered gc.disable() and never gc.enable().

The workaround we use is:
subprocess.gc.isenabled = lambda: True


The scenario for race condition is pretty obvious looking into the code below:

          gc_was_enabled = gc.isenabled() <- T1 gets false here
          gc.disable()
          try:
              self.pid = os.fork() <- while T2 is here
          except:
              if gc_was_enabled:
                  gc.enable()
              raise
          ... CODE FRAGMENT 1 ...
          if gc_was_enabled:
              gc.enable()

Also I wonder if exception fails in "... CODE FRAGMENT 1 ..." why don't we re-enable gc (finally block)

----------
messages: 269783
nosy: aonishuk
priority: normal
severity: normal
status: open
title: Race condition in subprocess.Popen which causes a huge memory leak
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27448>
_______________________________________