Python Heisenbugs? (was: Re: PyWart: The problem with "print")

[Devin Jeanpierre]

On Mon, Jun 3, 2013 at 12:34 AM, Dan Sommers <dan at tombstonezero.net> wrote:
> On Mon, 03 Jun 2013 13:37:27 +1000, Tim Delaney wrote:
>
>> With the increase in use of higher-level languages, these days
>> Heisenbugs most often appear with multithreaded code that doesn't
>> properly protect critical sections, but as you say, with lower-level
>> languages uninitialised memory is a common source of them.
>
> Aside from an I/O caching bug directly affected by calling print or
> somefile.write (where somefile is stdout), how else could I create a
> Heisenbug in pure Python?

The garbage collector can do this, but I think in practice it's
ridiculously rare, since __del__ is almost never useful due to its
unreliability*. The problem is that the garbage collector can do
whatever it wants. For example, in CPython it is called after so many
cycles have been created. This allows code and user actions to
inadvertently affect the garbage collector, and therefore, the
invocation of __del__.

If your __del__ does anything that accesses mutable global state also
used elsewhere, it's conceivable that the order of someone else's
access and __del__'s invocation depends on the GC. One order or the
other might be the wrong one which causes a failure. As it happens,
the "bt" command in pdb creates a cycle and might trigger the garbage
collector, causing __del__ to run immediately, and potentially hiding
the failure.

This isn't really "pure python" in that Python doesn't even guarantee
__del__ is ever called at all, let alone why. It's completely
implementation-specific, and not a property of Python the language.

-- Devin

.. [*] Some people use it as an "unreliable fallback"; this turns
their magical autosaving code into an attractive and yet horribly
dangerous nuisance. Friends don't let friends use __del__.