[Python-ideas] feature to make traceback objects usable without references to frame locals and globals

[M.-A. Lemburg]

Guido van Rossum wrote:
> On Sat, Jun 26, 2010 at 4:45 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Also note that garbage collection will not necessarily do what
>> the user expects: it is well possible that big amounts of
>> memory will stay allocated as unused space in pymalloc.
>> This is not specific to the discussed case, but still a valid
>> user concern. Greg Hazel observed this situation in his
>> example.
> 
> Aha. So whereas the process size ballooned, there is no actual memory
> leak (his example threw away the exception each time through the
> loop), it's just that looking at process size is a bad way to assess
> memory leaks. I would like to reject this then as "that's just how
> Python's memory allocation works". As you say, it's not specific to
> this case; it comes up occasionally and it's just a matter of user
> education.

pymalloc has gotten a lot better since it was fixed in Python 2.5
to return unused chunks of memory to the OS, but we still have the
issue of fragmented arenas with cases of just a few bytes
keeping 256kB (the size of an arena) allocated.

> I don't think anything should be done about __traceback__ either --
> frameworks that have this problem can work around it in various ways.
> Or, at least I don't see a reason to panic and roll back the feature.
> Maybe eventually it can be improved by adding some kind of
> functionality to control some details of the behavior.

Not necessarily roll back the feature, but an implementation
that deliberately introduces circular references is not really
ideal.

Since tracebacks on exceptions are rarely used by applications,
I think it would be better to turn them into weak references.

The arguments against doing this in the PEP appear rather
weak compared to the potential issue for non-expert Python
programmers:

"""
    Making the __traceback__ attribute a weak reference would avoid the
    problems with cyclic garbage.  Unfortunately, it would make saving
    the Exception for later (as unittest does) more awkward, and it would
    not allow as much cleanup of the sys module.
"""

Special use cases that want to save the traceback for later use
can always explicitly convert the traceback into a real (non-weak)
reference. I don't understand the reference to the sys module
cleanup, so can't comment on that.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jun 27 2010)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2010-07-19: EuroPython 2010, Birmingham, UK                21 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/