Persist a class (not an instance)

[David Wahler]

Kent Johnson wrote:
> Is there a way to persist a class definition (not a class instance,
> the actual class) so it can be restored later? A naive approach
> using pickle doesn't work:
[snip]
> The idea is to persist classes that are created and modified at runtime.

I couldn't resist the challenge, so I decided to take a crack at it. My
code is below. (I hope it's OK to post it even though it's a bit on the
long side.) So far, it seems to work OK; the biggest caveat to be aware
of is that functions' global context is not preserved.

My approach was to use pickle's __reduce__ protocol to store functions
and classes. Of course, you can't modify the built-in function and
classobj types, so I subclassed Pickler to override them. The advantage
of this is that you don't need an extension to the pickling data
format, and you can use the standard unpickler. (The custom module
still needs to have been imported, as it adds the classobj type to
__builtins__.)

Unfortunately, I'm not sure how to go about making it work for
new-style classes. It would seem to involve messing with dictproxy and
descriptor objects, and that's getting me into more unfamiliar
territory.

I'm sure there's a better way to do this; this seemed like "the
simplest thing that could possibly work".

-- David

#####################################################
# code_pickle.py

import sys, copy_reg, pickle, new, marshal, types, StringIO

# Needed to unserialize old-style classes
sys.modules['__builtin__'].classobj = new.classobj

# from http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/439096
def get_cell_value(cell):
    return type(lambda: 0)(
        (lambda x: lambda: x)(0).func_code, {}, None, None, (cell,)
    )()

def func_constructor(name, code, defaults, closure):
    return new.function(marshal.loads(code), globals(), name,
            defaults, closure)

class CodePickler(pickle.Pickler):
    def __init__(self, *args, **kwargs):
        pickle.Pickler.__init__(self, *args, **kwargs)
        self.dispatch = self.dispatch.copy()
        self.dispatch[types.ClassType] = CodePickler.do_class
        self.dispatch[types.FunctionType] = CodePickler.do_function

    def save(self, ob, *args, **kwargs):
        print ob
        pickle.Pickler.save(self, ob, *args, **kwargs)

    def do_class(self, ob):
        if ob in (types.__dict__.values()):
            self.save_global(ob)
        else:
            args = (ob.__name__, ob.__bases__, ob.__dict__)
            self.save_reduce(type(ob), args)

    def do_function(self, ob):
        if ob == func_constructor:
            self.save_global(ob)
        else:
            if ob.func_closure:
                closure = tuple(map(get_cell_value, ob.func_closure))
            else:
                closure = None
            args = (ob.func_name, marshal.dumps(ob.func_code),
                    ob.func_defaults, closure)
            self.save_reduce(func_constructor, args)

def dumps(ob):
    s = StringIO.StringIO()
    CodePickler(s).dump(ob)
    return s.getvalue()

# Example:
#
# import code_pickle
# class Foo:
#     def show(self):
#         print "Foo!"
#
# s = code_pickle.dumps(Foo)
# --------------------------------------
# import code_pickle, pickle
# Foo = pickle.loads(s)
# Foo().show
#####################################################