copy_reg problem - UPDATE! PLEASE READ!

[Rainer Deyke]

"Martin von Loewis" <loewis at informatik.hu-berlin.de> wrote in message
news:p6q8zrzz4hr.fsf at informatik.hu-berlin.de...
> "Rainer Deyke" <root at rainerdeyke.com> writes:
>
> > Appearantly copy_reg.pickle only works for types, not classes (despite
the
> > documentation saying otherwise).
>
> I'd say the documentation is wrong, not the code.
>
> > I really need this to work for class instances for two reasons:
> >
> >   1. It allows me to later replace a class with a C extension type.
>
> Can you please elaborate? What is it that you want to do, and why is
> that a good thing?

class A:
  ...

def create_A(data):
  return A(data)

def pickle_A(a):
  return create_A, (a.get_data(),)

copy_reg.pickle(A, pickle_A, create_A)

I can later move the implementation of A into a C extension type and still
be able to load old pickles containing A by replacing the create_A function
(which is pickled by name).  This is important, because my current project
may require better performance than Python can provide.

> >   2. It allows me to "merge" objects from different pickles.
>
> Again, I don't understand. In what way do you want to merge objects
> from different pickles, and why is there no other way to achieve the
> same effect?

I have a variety of interconnected persistant objects which can't all be
loaded into memory at the same time.  Therefore I want pickled objects that
when loaded connect to the objects that are already in memory.  I wrote a
class to facilitate this (shown in simplified form):

class ResourceDescriptor:
  def __init__(self, name):
    self.name = name
    self.object = None
  def get(self):
    if self.object is None:
      self.object = load_resource(self.name)
      # load_resource ultimately loads a pickle from a database
    return self.object

resource_descriptors = {}

def get_resource_descriptor(name):
  if not resource_descriptors.has_key(name):
    resource_descriptor[name] = ResourceDescriptor(name)
  return resource_descriptors[name]

# There is also a function to delete resource descriptors that are
# no longer needed.

Resource objects reference other resource objects indirectly through
references to ResourceDescriptor instances.  However, the system relies on
ResourceDescriptor instances being pickled by name, not state: otherwise I
end up with duplicate objects loaded, which is unacceptable. (The full
implementation of ResourceDescriptor allow mutable resources, which are
written back to the database before being unloaded.)  I could approach this
problem by transforming ResourceDescriptor instances to strings and vice
versa in the resourc objects' __getstate__ and __setstate__ methods, but
this approach is messy.

A more simple example would a singleton:

class S:
  pass

the_S = S()

def get_the_S():
  return the_S

def pickle_S(s):
  return get_the_S, ()

copy_reg.pickle(S, pickle_the_S, get_the_S)


--
Rainer Deyke (root at rainerdeyke.com)
Shareware computer games           -           http://rainerdeyke.com
"In ihren Reihen zu stehen heisst unter Feinden zu kaempfen" - Abigor