[Python-Dev] What is the precise problem? [was: Reference cycles in Exception.__traceback__]

[Jim J. Jewett]

On Mon Mar 10 18:56:17 CET 2014 (and earlier quotes), Maciej Fijalkowski wrote:

Maciej:>> You should not rely on __del__s being called timely one way or
Maciej:>> another. Why would you require this for the program to work
Maciej:>> correctly in the particular example of __traceback__?

To the extent that I understand, he isn't requiring it for correctness;
he is instead saying that without timely __del__, the Quality of
Implementation suffers.

I suspect there are aspects of tulip (or event processing in general)
that make it more common for the frame graph to be painfully cyclic,
so that live frames keep dead ones from being collected.

It may also be more common to have multiple __del__ methods in the
same cycle, if cycles are created by a framework.  So the problems
aren't new, but they may have become considerably more painful.

Victor:> For asyncio, it's very useful to see unhandled exceptions as early as
Victor:> possible. Otherwise, your program is blocked and you don't know why.

...

Maciej:>> twisted goes around it by attaching errback by hand. Would that work
Maciej:>> for tulip?

Maciej:>> deferred.addErrback(callback_that_writes_to_log)

What do you mean "by hand"?  Does the framework automatically add a
"log the exception" errback to every task, or every task that doesn't
have its own errback of some sort?  Or do you mean that users should
do so by hand, but that it is a well-known recipe?


Maciej:>> I'm very skeptical about changing details of __traceback__ and
Maciej:>> frames, just in order to make refcounting work (since it would
Maciej:>> create something that would not work on pypy for example).

How about just loosening some constraints on exceptions, in order to
permit more efficient operation, but in a way that may be particularly
useful to a refcounting scheme?  Can I assume that you don't object to
frame.clear()?  How about a hypothethetical traceback.pack() that
made it easier to reclaim memory held by frame/traceback cycles?
If standard traceback printing were the only likely future use, each
frame/traceback pair could be replaced by 4 pointers, and allocating
space/copying those 4 would be the only work that wasn't already needed
for the eventual deallocation.

Today, the setters for __cause__, __context__, and __traceback do
typechecks to ensure that those properties are (None or) the
expected type; __traceback__ doesn't even allow subclasses.  The
constructors for frame and traceback are similarly strict.

What would be the harm in allowing arbitrary objects, let alone
a few specific alternative implementations?

-jJ

--

If there are still threading problems with my replies, please
email me with details, so that I can try to resolve them.  -jJ