[issue26789] logging: Trying to log during Python finalization with NameError: name 'open' is not defined

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

I fixed the issue in the master and I will not backport the change to stable branches on purpose. Thanks for the review Vinay!

--

I pushed a change in Python 3.10 which reduces the risk of "NameError: name 'open' is not defined" when logging really late during the Python finalization.

It's a workaround on fragile finalization code. Please don't rely on it! You should release resources explicitly at exit. Don't rely on implicit __del__() finalizers.

I chose to enhance the logging module anyway since the initial issue was that asyncio failed to log a message at exit, a message which is supposed to help developer to fix their code. It's a convenient tool to ease debugging. But, again, it would be really bad idea to rely on it to ensure that logging still works after logging.shutdown() has been called!

About asyncio, for development, you can try to frequently call gc.collect() using call_later() or something, to get issues like "never-retrieved exceptions":
https://docs.python.org/dev/library/asyncio-dev.html

I chose to restrict this issue to the very specific case of NameError when calling logging.FileHandler.emit() late during the Python finalization.

If someone wants to enhance another function, please open a new issue.

I will not backport my change since the Python finalization is really fragile and my logging fix rely on many enhancements made in the master that will not be backported to Python 3.9 and older on purpose. Changes on the finalization are also fragile and can introduce new subtile bugs. It happened multiple times. But overall, step by step, the Python finalization becomes more and more reliable ;-)

For curious people, here are my notes on the Python finalization:
https://pythondev.readthedocs.io/finalization.html

--

About the logging, honestly, I'm not sure that it's a good idea that FileHandler.emit() reopens the file after logging.shutdown() has been called (this function calls FileHandler.close() which closes the file).

But if someone disagrees, please open a separated issue ;-) This one is now closed.

----------
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue26789>
_______________________________________