Unpickling data with classes from dynamic modules

[Fredrik Tolf]

Dear list,

I have a system in which I load modules dynamically every now and then 
(that is, creating a module with types.ModuleType, compiling the code for 
the module and then executing it in the module with exec()), and where I 
would wish to be able to have classes in those modules containing classes 
that could pickled, and then unpickled again.

The problem with that, currently, is of course two-fold: The dynamically 
loaded module may not be loaded at the time when the unpickling takes 
place, but even if it were, it isn't registered in sys.modules, so the 
unpickler can't find it either way. And, of course, that is detected 
already when pickling.

Is there any way to fix this, at all?

I considered trying to create subclasses of the pickler and unpickler that 
pickle a reference to a module loader and data for the particular module 
along with a class that comes from such a module, by overriding 
Pickler.save_global and Unpickler.load_global. Unfortunately, however, 
those functions aren't part of the public interface and can't be 
overridden when the C-pickle module is used instead (which, obviously, is 
what normally happens).

Is there any way around this without having to modify the pickle module 
itself?

--

Fredrik Tolf