Unpickling data with classes from dynamic modules

[dieter]

Fredrik Tolf <fredrik at dolda2000.com> writes:

> ...
>> Maybe all you need to add is implementation for obj.__reduce__
>
> That would certainly work, and I guess I could perhaps use it as a
> work-around, but that would mean I'd have to mark every single such
> class as such in some way or another. What I'm looking for is a
> solution that could make them picklable transparently, just like
> normal.

This is not (easily) possible. In the "normal/transparent" case,
"pickle" treats the class/type part of a class/type instance
(unless it explicitely knows the "class/type" as requiring special
treatment) as a "global" (essentially a module path and a name)
and it expects that "global"s can simply be imported.


The ZODB ("Zope Object DataBase") is build on top of "pickle"
("cpickle", in fact) - and it provides for an application level
module loader. Zope is using this feature to implement
so called "ZClasses", classes the code of which are maintained
inside the ZODB.

Thus, this is a case similar to yours
(your dynamic modules correspond to code maintained in the ZODB).
This implies that there are ways to achieve something like this
(in principle) and you may look how it is done in Zope to
get some ideas. However, it is likely to be difficult.
Your "module loader" must be able to recreate your dynamic
modules - for this, it needs all the creation input data.
In the Zope case, this comes from the ZODB. You would need
to find a way to pass this information to the unpickling process...