Shelve doesn't appear to restore references properly????

Muppet google.com at magicdb.com
Sat Mar 8 17:59:49 EST 2003


Ok. This is a little fiddly to describe, so I'm going to describe it
and then show some sample code.

The problem is that references between objects do not appear to work
properly in shelve. When restoring from a shelve, objects which are
shared in the original data appear to come back unshared - that is,
where there was one object originally, after being restored from a
shelf there are many identical copies.

First, the ordinary case:
1> Take some objects which have sub-objects in common.
2> In the normal, un "shelved" state, modify one of the sub-objects by
(say) assigning it a new value.
3> Objects which share that sub-object reflect that change: they all
reference the same object.

Now, the shelve case:
1> Take some objects which have sub-objects in common.
2> Shelve these objects.
3> Unshelve these objects: restore them into memory from disk.
4> Modify one of the sub-objects by (say) assigning it a new value.
5> Objects which share that sub-object reflect **do**not** that
change: they now all reference their own copies of that sub-object.

Now, I'm trying to figure this out: is that the correct behavior? Am I
doing something stupid (source code below)? Is there any easy way of
restoring state *fully*, so that information encoded as which objects
are shared between other objects is also restored?

Here's the demo code:
#!/usr/bin/python
import shelve
import cPickle

howmany = 10

class muppet(object):
	""" a muppet """
	def set(self, value):
		self.value = value
	def __init__(self, value):
		self.set(value)
		self.submuppet = []
	def disp(self, prefix = ""):
		print prefix + "Muppet Value = ", self.value
		for x in self.submuppet:
			x.disp(prefix + "---")
	def addsubmuppet(self,submuppet):
		self.submuppet.append(submuppet)

filename = "test.data"

#the shared submuppet
submup = muppet(34)

#generate a group of muppets which all share a single submuppet.
#save them to a shelf.
d = shelve.open(filename)
for x in xrange(howmany):
        mup = muppet(x)
        mup.addsubmuppet(submup)
        d[str(x)] = mup
d.close()


#generate a group of muppets which all share a single submuppet.
print "Muppets held in memory:"
for x in xrange(howmany):
	mup = muppet(x) 
        mup.addsubmuppet(submup)
#half way through, modify the value of the submuppet
	if mup.value == 5 :
		mup.submuppet[0].value = 99
	mup.disp()


#now restore the muppets from the shelf 
#and modify the submuppet half way through the restore.
#it appears that the restored muppets do not reflect that change
#implying that they do not share the submuppet any more.
#Each has it's own submuppet now.

print 
print "Muppets restored from a shelf:"
d = shelve.open(filename)
d['34'] = submup #shelve the submuppet (same result if it is not
shelved)
mlist = []
for key in d.keys():
	data = d[key]
	mlist.append(data)
	if data.value == 5 :
		data.submuppet[0].value = 99
	data.disp()
d.close();

#once restored, is there any difference?
print
print "Restored muppets still don't work properly"
for data in mlist:
	if data.value == 5 :
		data.submuppet[0].value = 99
	data.disp()

#Somebody please let me know if I'm smoking crack here? Are my
expectations of Shelve #unreasonable?!?!?!?!




More information about the Python-list mailing list