[issue6721] Locks in python standard library should be sanitized on fork

[lesha]

lesha <pybug.20.lesha at xoxy.net> added the comment:

I am really alarmed by the reinit_locks patches.

I scanned the comment history, and looked at the patch. I may have missed something, but it looks to me like the basic behavior is this:

"After fork(), all locks are replaced by brand-new lock objects that are NOT locked."

*Grim Prediction*: This is going to cause some disastrous, unexpected, and hilarious data loss or corruption for somebody.

Here is why:

  class MySQLConn:

    def __init__(self):
      self.lock = Lock()

    def doWork(self):
      self.lock.acquire()
      # do a sequence of DB operations that must not be interrupted,
      # and cannot be made transactional.
      self.lock.release()

Run this in a thread:

  def thread1(conn):
    while True:
      conn.doWork()
      time.sleep(0.053)

Run this in another thread:

  def thread2(conn):
    while True:
      conn.doWork()
      time.sleep(0.071)

Run this in a third thread:

  def thread2():
    while True:
      subprocess.call(["ls", "-l"])
      time.sleep(0.3)

With reinit_locks(), this will eventually break horribly. 

a) fork() is called with the DB lock held by thread1.
b) Some time passes before the child gets to exec().
c) In that time, the child's thread2 gets to doWork(). 
d) Simultaneously, the parent's doWork is still running and holding a lock.
e) Thanks to reinit_locks, the child's thread2 does not have a lock, and it will merrily proceed to corrupt the parent's work.

So I think this approach is basically doomed.

I think my approach of marking _some_ locks as safe to reinit upon fork is workable (i.e. to solve the bad interaction with logging or import). 

However, there's also an orthogonal approach that might work well:

1) Right before the first thread gets created in any Python program, fork off a fork() server. 

>From then on, subprocess will only use the fork server to call commands.

Thoughts?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6721>
_______________________________________