[issue40091] Crash in logging._after_at_fork_child_reinit_locks()

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

> because I encountered a problem where a standard library lock was held by a parent process at the time that child processes were forked, so the child processes got stuck behind the inherited held locks.

Which lock from which module? You wrote details at:
https://github.com/python/cpython/pull/19195#issuecomment-609583084

According to your comment #28 at https://bugs.launchpad.net/nova/+bug/1844929 the lock involved in the issue comes from _TransactionFactory of oslo.db:
https://github.com/openstack/oslo.db/blob/b903d4e1ee07ef2ec454daa5b8418b3039e02774/oslo_db/sqlalchemy/enginefacade.py#L189

So it's a bug in oslo.db, not in Python. Python doesn't provide any machinery to automatically reinitialize all locks created in Python at fork in the child process. os.register_at_fork() must be used explicitly.


> But, if I'm understanding correctly, this issue is fixing something in python logging specifically and not all standard library locks in general.

This issue is specific to logging.


> My question is, will there be a way to reinit standard library locks in general using _at_fork_reinit()? That is, should we expect a future fix in python to do this or is the recommendation to continue to ensure the application reinits locks during process start if we know the process could be a child?

Each module has to setup an os.register_at_fork() callback to reinitialize its locks. It's done by threading and logging modules for example. The multiprocessing has its own util.register_after_fork() machinery (see bpo-40221)..

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40091>
_______________________________________