[issue6721] Locks in python standard library should be sanitized on fork

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> Yes, we would need to keep track of the thread id and process id inside
> the lock. We also need a global variable of the main thread id after
> fork, and a per-lock "taken" flag.
>
> Synopsis:
>
>    def _reinit_if_needed(self):
>        # Call this before each acquire() or release()
>        if self.pid != getpid():
>            sem_init(self.sem, 0, 1)
>            if self.taken:
>                if self.tid == main_thread_id_after_fork:
>                    # Lock was taken in forked thread, re-take it
>                    sem_wait(self.sem)
>                else:
>                    # It's now released
>                    self.taken = False
>            self.pid = getpid()
>            self.tid = current_thread_id()
>

A couple remarks:
- with linuxthreads, different threads within the same process have
the same PID - it may be true for other implementations - so this
would lead to spurious reinitializations
- what's current_thread_id ? If it's thread_get_ident (pthread_self),
since TID is not guaranteed to be inherited across fork, this won't
work
- calling getpid at every acquire/release is expensive, even though
it's a trivial syscall (it'll have to measured though)
- imagine the following happens:

P1

lock.acquire()
fork()    ->       P2
                       start_new_thread T2
                       T1           T2
                                        lock.acquire()

The acquisition of lock by T2 will cause lock's reinitialization: what
happens to the lock wait queue ? who owns the lock ?
That why I don't think we can delay the reinitialization of locks, but
I could be wrong.

> Well, I fail to understand how that idiom can help us. We're not a
> self-contained application, we're a whole programming language.
> Calling fork() only when no lock is held is unworkable (for example, we
> use locks around buffered I/O objects).

Yes, but in that case, you don't have to reacquire the locks after fork.
In the deadlock you experienced above, the thread that forked wasn't
the one in the I/O code, so the corresponding lock can be
re-initialized anyway, since the thread in the I/O code at that time
won't exist after fork.
And it's true with every lock in the library code: they're only held
in short critical sections (typically acquired when entering a
function and released when leaving), and since it's not the threads
inside those libraries that fork, all those locks can simply be
reinitialized on fork, without having the reacquire them.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6721>
_______________________________________