[Python-ideas] feature to make traceback objects usable without references to frame locals and globals

[Brett Cannon]

On Sun, Jun 27, 2010 at 10:20, M.-A. Lemburg <mal at egenix.com> wrote:
> Guido van Rossum wrote:
>> On Sat, Jun 26, 2010 at 4:45 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> Also note that garbage collection will not necessarily do what
>>> the user expects: it is well possible that big amounts of
>>> memory will stay allocated as unused space in pymalloc.
>>> This is not specific to the discussed case, but still a valid
>>> user concern. Greg Hazel observed this situation in his
>>> example.
>>
>> Aha. So whereas the process size ballooned, there is no actual memory
>> leak (his example threw away the exception each time through the
>> loop), it's just that looking at process size is a bad way to assess
>> memory leaks. I would like to reject this then as "that's just how
>> Python's memory allocation works". As you say, it's not specific to
>> this case; it comes up occasionally and it's just a matter of user
>> education.
>
> pymalloc has gotten a lot better since it was fixed in Python 2.5
> to return unused chunks of memory to the OS, but we still have the
> issue of fragmented arenas with cases of just a few bytes
> keeping 256kB (the size of an arena) allocated.
>
>> I don't think anything should be done about __traceback__ either --
>> frameworks that have this problem can work around it in various ways.
>> Or, at least I don't see a reason to panic and roll back the feature.
>> Maybe eventually it can be improved by adding some kind of
>> functionality to control some details of the behavior.
>
> Not necessarily roll back the feature, but an implementation
> that deliberately introduces circular references is not really
> ideal.

But the circular reference only occurs if you store a reference
outside the 'except' clause; Python 3 explicitly deletes any caught
exception variable to prevent the loop.

>
> Since tracebacks on exceptions are rarely used by applications,
> I think it would be better to turn them into weak references.
>

While I would be fine with that if that many people save raised
exceptions outside of an 'except' clause, I doubt that happens very
often and there would be backward-compatibility issues at this point.