Python object persistency using pickle...

[Greg Ewing]

David CROSSON wrote:
> 
>     pickle.dump(b,f3)
>     pickle.dump(a1,f1)
>     pickle.dump(a2,f2)

That doesn't work because pickle.dump creates a new Pickler 
instance every time you call it. To preserve references among 
objects, all the objects involve have to be pickled using a 
*single* Pickler instance.

One way to achieve that is to arrange for all your objects
to be reachable from a single "root" object and pickle that
object using a single call, e.g.

   pickle.dump((a1, a2, b), f1)

and to restore it,

   a1, a2, b = pickle.load(f1)

Another is to explicitly create your own Pickler instance:

   p = pickle.Pickler(f1)
   p.dump(a1)
   p.dump(a2)
   p.dump(b)

and similarly restore using a singe Unpickler:

  u = pickle.Unpickler(f1)
  a1 = u.load()
  a2 = u.load()
  b = u.load()

Both those methods will store all three objects together in
one file, which is probably desirable because they are so
closely coupled.

If you really want them in separate files, you might be able
to pull some trick like switching the file attached to the
Pickler instance in between calls, but I'm not sure what that
would gain you. You would still have to make sure you unpickled
them in exactly the same order that you pickled them, so you
might as well just put them in the same file.

If you want random access to pickled objects, the shelve module
might help -- although I don't know what it does with references
between separately shelved objects. Anyone else know more?

Greg