copy_reg problem - UPDATE! PLEASE READ!

[Martin von Loewis]

"Rainer Deyke" <root at rainerdeyke.com> writes:

> > Can you please elaborate? What is it that you want to do, and why is
> > that a good thing?
> 
> class A:
>   ...
> 
> def create_A(data):
>   return A(data)
> 
> def pickle_A(a):
>   return create_A, (a.get_data(),)
> 
> copy_reg.pickle(A, pickle_A, create_A)
> 
> I can later move the implementation of A into a C extension type and still
> be able to load old pickles containing A by replacing the create_A function
> (which is pickled by name).  This is important, because my current project
> may require better performance than Python can provide.

You can do that without copy_reg. If you look at the pickle code,
you'll find that restoring a class does

  args=
  module=
  name=
  klass = find_class(module, name)
  inst = apply(klass, args)
  state =
  inst.__setstate__(state)

So if you later change class A to a type, *and* you still need to be
able to read old pickles, just arrange that find_class will return
something meaningful, e.g. a function to create instances of your
type. It is then your choice whether the type supports a __setstate__,
or updating of  __dict__, or item-by-item update of its state
members - see the source of pickle for the exact algorithm used.

> I have a variety of interconnected persistant objects which can't
> all be loaded into memory at the same time.  Therefore I want
> pickled objects that when loaded connect to the objects that are
> already in memory.  I wrote a class to facilitate this (shown in
> simplified form):

I believe your scheme can be greatly simplified by using the
persistent_id mechanism of pickle. I.e. assign functions to the
persistent_id (or inst_persistent_id) attributes of your pickler
object (works both for pickle and cPickle). Then use your resource
names as persistent ids.

Regards,
Martin