Automatic reloading, metaclasses, and pickle

[andrewfelch at gmail.com]

On Feb 27, 6:47 pm, "Ziga Seilnacht" <ziga.seilna... at gmail.com> wrote:
> Andrew Felch wrote:
> > Thanks for checking.  I think I narrowed the problem down to
> > inheritance.  I inherit from list or some other container first:
>
> > class PointList( list, AutoReloader ):
> >     def PrintHi1(self):
> >         print "Hi2"
>
> > class MyPrintingClass( AutoReloader ):
> >     def PrintHi2(self):
> >         print "Hi2v2"
>
> > Automatic reloading works for MyPrintingClass but not for PointList.
> > Any ideas?
>
> > -Andrew
>
> Ah yes, this is the problem of list.__new__ not calling the next
> class in MRO. Try to switch the bases, so that AutoReloader's
> __new__ method will be called first.
>
> Ziga- Hide quoted text -
>
> - Show quoted text -

So there is another bug crippling Python (see
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=628925&group_id=5470
).  Pickle and cPickle get confused after reloading, and won't let you
dump the objects to file using protocol 2.  The bug occurred for me
when an instance of an AutoReloader class had a member not derived
from AutoReloader (error text: "it's not the same object" ).  The
workaround is to use text mode, which I assume means not using
protocol 2.  After that, you can reload them in text mode, and then
dump them using protocol 2.  This is not so bad since you only do it
when you were going to dump objects to file anyway, so it really only
takes 3x as long on those occassions.  The metaclass adaptation is
still incredibly useful because it avoids having to do pickle dump-
loads when you wouldn't otherwise have had to (approaching an infinite
speedup in those cases, thanks Ziga! :-).

Another caveat is that pickle dumping doesn't work if there is a
pointer to a global function (that changes) that is a member of an
instance of a class derived from AutoLoader.  The error message has
subtext: "it's not the same object ".  That doesn't seem so bad to me,
since you don't have to do that type of thing (function pointers) so
much now that you can use classes!

The best solution would be to make pickle protocol 2 have the same
behavior as the text protocol (i.e. actually work in this case).

Regarding the function pointers, anybody (Ziga :-) know why it is that
pickle chokes so hard on this?  Is there some adaptation of the
metaclass or a less beautiful hack that could fix it?

Thanks to those readers following me down the rabbit hole.  Ziga's
metaclass adaptation has saved Python in my mind, and these nits seem
to be worth discussing.

- Andrew Felch