How much is set in stone?

[Barry A. Warsaw]

[Andrew posts a cute way to trick cPickle into deleting a file...]

Note that pickle isn't so easily tricked, but only because of an
implementation accident in TemporaryFileWrapper.  This was a fun one
that had Tim and me stumped for a bit while we tried to figure out
exactly what was going on.

Turns out that if you use pickle for this example, you'll get an
exception during loads() because TemporaryFileWrapper implements a
__getattr__() that fails when the Unpickler tries to see if the object
has a __setstate__ attribute.  This is because the instance's __dict__
doesn't yet have a 'file' key, so you get a KeyError at that point.

cPickle is vulnerable because it masks all exceptions during the
search for inst.__setstate__; if that fails it just interprets that to
mean that it had no __setstate__ attribute and continues onward.
pickle OTOH only catches AttributeErrors so the KeyError shorts out
pickle.loads() and thus your TemporaryFileWrapper instance never gets
a chance to run its __del__().

    AD> Sure, and it was doable in pickle.py as well by making a
    AD> subclass and overriding the find_class method (example shown
    AD> elsewhere).  But I didn't know I needed to do that, and it's
    AD> hard to figure out which classes are okay to pickle safely.
    AD> Some of my own classes have these sorts of filesystem cleanup
    AD> code in __del__.  So pickles are hard to use safely for
    AD> insecure-but-non-trivial data.

Indeed.  I'm curious on how we can systematically make it better, but
that's for another day, hopefully to be captured in a pickle PEP for
2.3.

sniffing-pickles-is-bad-for-your-health-ly y'rs,
-Barry