Automatic reloading, metaclasses, and pickle

[andrewfelch at gmail.com]

On Feb 27, 5:30 pm, "Ziga Seilnacht" <ziga.seilna... at gmail.com> wrote:
> Andrew Felch wrote:
>
> > Thanks Ziga.  I use pickle protocol 2 and binary file types with the
> > command: "cPickle.dump(obj, file, 2)"
>
> > I did your suggestion, i commented out the "__call__" function of
> > MetaInstanceTracker and copied the text to the __new__ function of
> > AutoReloader (code appended).  I got a crazy recursive error message
> > (also appended below).  In my code, I am creating a new instance,
> > rather than using the pickled object (it needs to work in both modes).
>
> > Thanks very much for helping me get through this.  With my development
> > approach, finding a solution to this problem is really important to
> > me.
>
> Here is a version that should work. It should work with all protocols,
> see InstanceTracker.__reduce_ex__. Note that all subclasses of
> InstanceTracker and AutoReloader should be cautious when overriding
> the
> __new__ method. They must call their base class' __new__ method,
> preferably by using super(), or the tracking won't work.
>
> # adapted fromhttp://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/160164
>
> import weakref, inspect
>
> class MetaInstanceTracker(type):
>
>     def __init__(cls, name, bases, ns):
>         super(MetaInstanceTracker, cls).__init__(name, bases, ns)
>         cls.__instance_refs__ = []
>
>     def __instances__(cls):
>         instances = []
>         validrefs = []
>         for ref in cls.__instance_refs__:
>             instance = ref()
>             if instance is not None:
>                 instances.append(instance)
>                 validrefs.append(ref)
>         cls.__instance_refs__ = validrefs
>         return instances
>
> class InstanceTracker(object):
>
>     __metaclass__ = MetaInstanceTracker
>
>     def __new__(*args, **kwargs):
>         cls = args[0]
>         self = super(InstanceTracker, cls).__new__(*args, **kwargs)
>         cls.__instance_refs__.append(weakref.ref(self))
>         return self
>
>     def __reduce_ex__(self, proto):
>         return super(InstanceTracker, self).__reduce_ex__(2)
>
> class MetaAutoReloader(MetaInstanceTracker):
>
>     def __init__(cls, name, bases, ns):
>         super(MetaAutoReloader, cls).__init__(name, bases, ns)
>         f = inspect.currentframe().f_back
>         for d in [f.f_locals, f.f_globals]:
>             if name in d:
>                 old_class = d[name]
>                 for instance in old_class.__instances__():
>                     instance.change_class(cls)
>
> cls.__instance_refs__.append(weakref.ref(instance))
>                 for subcls in old_class.__subclasses__():
>                     newbases = []
>                     for base in subcls.__bases__:
>                         if base is old_class:
>                             newbases.append(cls)
>                         else:
>                             newbases.append(base)
>                     subcls.__bases__ = tuple(newbases)
>                 break
>
> class AutoReloader(InstanceTracker):
>
>     __metaclass__ = MetaAutoReloader
>
>     def change_class(self, new_class):
>         self.__class__ = new_class
>
> Ziga

I pasted the code into mine and replaced the old.  It seems not to
work for either unpickled objects or new objects.  I add methods to a
class that inherits from AutoReloader and reload the module, but the
new methods are not callable on the old objects.  Man!  It seems we're
so close, it will be huge if this ends up working.  This stuff is so
over my head, I wish I could help in some additional way.

-Andrew