[Python-Dev] tracemalloc.get_object_traceback() and "unclosed" ResourceWarning

[Victor Stinner]

Hi,

Since Python 3.3, when a file or socket object is destroyed before
being closed, a ResourceWarning is emitted. The warning is emitted
where the object is destroyed: it's common to see "gc.collect()"
location in the Python test suite for example.

Tarek Ziadé asked on Twitter if there is a tool to track the origin of
the object:
"I wonder is there's a tool to trace back objects declarations that
raise ResourceWarning in Py"
https://twitter.com/tarek_ziade/status/394006754779877377

Nick Coghlan ‏replied that tracemalloc may help:
"@tarek_ziade tracemalloc (http://www.python.org/dev/peps/pep-0454/ …
- not accepted yet) will likely make it possible in 3.4, dunno about
earlier versions."
https://twitter.com/ncoghlan_dev/status/394010756242350080

I checked the code and it's not so simple. FileIO destructor emits the
warning (fileio_dealloc_warn() in Modules/_io/fileio.c), but it's not
possible to "hook" this function. I tried something with weakref, but
I failed to set my hook.

Does anyone see how to reuse tracemalloc (which is optional and
disabled by default) in FileIO destructor?

The file/object destructor may retrieve somehow the most recent frame
of the object and call PyErr_WarnExplicitObject() instead of
PyErr_WarnFormat() to pass the filename and lineno of the object. But
how would Python decide to use tracemalloc or not? Add a new genric
"get object traceback" callback which would be set by tracemalloc when
the module is enabled? A frame is maybe not enough, it would be better
to get the whole traceback!

Do you know other features where it would be nice to know the origin
of a Python object?

Victor