[Python-Dev] bpo-36829: Add sys.unraisablehook()

[Victor Stinner]

Le jeu. 16 mai 2019 à 17:56, Steve Dower <steve.dower at python.org> a écrit :
> I really like this API, and I agree with Victor that we don't really
> need more than the exception info. For future expansion, we can pass in
> a different exception, no?

Sorry, I don't understand. I explained that we need more than
(exc_type, exc_value, exc_tb).

"obj" is part of the C function PyErr_WriteUnraisable(). We have to
pass it to the hook. Otherwise, how do you want to guess where the
exception comes from? Without any context, it can be super painful to
debug such exception :-(

That's why I said that I like Serhiy's idea of extending the API to
allow to also pass an error message.

Unraisable exceptions are commonly raised during garbage collection or
in finalizers. So it's commonly happens when you don't "expect" them
:-)


> I'm not even convinced we need the obj argument, or that it's a good
> idea - this is yet another place where it's likely to be dead/dying
> already. And what can you do with it? Resurrection seems like a really
> bad idea, as does diving into a custom __repr__. There's no useful
> recovery mechanism here that I'm aware of, so I'd be in favor of just
> passing through the exception and nothing else.

Well, first of all, I would advice to *not* keep "obj" alive after the
execution of the hook. Keep repr(obj) if you want, but not a reference
to the object. Same for the exception value, keeping it alive is a
high risk of creating annoying reference cycles ;-)

--

In a finalizer, "obj" is not always the object being finalized.

Example: when an _asyncio.Future is finalized,
loop.call_exception_handler() is called if the future wasn't consumed.
If this call fails, PyErr_WriteUnraisable(func) is called. func isn't
the future, but the call_exception_handler() method.

Example: on a garbage collection, callbacks of weak references are
called. If a callback fails, PyErr_WriteUnraisable(callback) is
called. It's not the collected object nor the weak reference.

It's also common that PyErr_WriteUnraisable(NULL) is called. In this
case, you have no context where the exception comes from :-( For that,
I experimented a custom hook which logs also the current Python stack.
That's useful in many cases!

socket.socket finalizer is a case where the finalized object (the
socket) is passed to the hook. IMHO it's fine to resurrect a socket in
that case. Finalization isn't deallocation. The objet remains
consistent. The finalization protocol ensures that the finalizer is
only called once.

In practice, I wouldn't advice to resurrect objects :-) I would expect
that a hook only calls repr(hook) and then forgets the object.

The current PyErr_WriteUnraisable() implementation does exactly that:
it formats repr(obj) into sys.stderr.

*If* someone finds a case where PyErr_WriteUnraisable() can resurect
an object that is freed just after the call, I would suggest to fix
the code to not pass the object to PyErr_WriteUnraisable(). Using PEP
442 finalizers, I don't think that it should happen.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.