[Python-Dev] Crash in new "trashcan" mechanism.

[Mark Hammond]

The trashcan bug turns out to be trivial to describe, but not so
trivial to fix.  Put simply, the trashcan mechanism conflicts
horribly with PY_TRACE_REFS :-(

The problem stems from the fact that the trashcan resurrects
objects.  An object added to the trashcan has its ref count as zero,
but is then added to the trash list, transitioning its ref-count
back to 1.  Deleting the trashcan then does a second deallocation of
the object, again taking the ref count back to zero, and this time
actually doing the destruction.  By pure fluke, this works without
Py_DEBUG defined!

With Py_DEBUG defined, this first causes problems due to ob_type
being NULL.  _Py_Dealloc() sets the ob_type element to NULL before
it calls the object de-allocater.  Thus, the trash object first hits
a zero refcount, and its ob_type is zapped.  It is then resurrected,
but the ob_type value remains NULL.  When the second deallocation
for the object happens, this NULL type forces the crash.

Changing the Py_DEBUG version of _Py_Dealloc() to not zap the type
doesnt solve the problem.  The whole _Py_ForgetReference()
linked-list management also dies.  Second time we attempt to
deallocate the object the code that removes the object from the
"alive objects" linked list fails - the object was already removed
first time around.

I see these possible solutions:

* The trash mechanism is changed to keep a list of (address,
deallocator) pairs.  This is a "cleaner" solution, as the list is
not considered holding PyObjects as such, just blocks of memory to
be freed with a custom allocator.  Thus, we never end up in a
position where a Python objects are resurrected - we just defer the
actual memory deallocation, rather than attempting a delayed object
destruction.  This may not be as trivial to implement as to describe
:-)

* Debug builds disable the trash mechanism.  Not desired as the
basic behaviour of the interpreter will change, making bug tracking
with debug builds difficult!  If we went this way, I would (try to
:-) insist that the Windows debug builds dropped Py_DEBUG, as I
really want to avoid the scenario that switching to a debug build
changes the behaviour to this extent.

* Perform further hacks, so that Py_ForgetReference() gracefully
handles NULL linked-list elements etc.

Any thoughts?

Mark.