changes to shelf items fail silently?

[Steven Taschuk]

Quoth eichin at metacarta.com:
>
> > They're mutating the returned list (to which you retain no
> > references, and which therefore is immediately discarded).
> 
> Ah, so I'm missing the distinction that it's returning a list, and not
> being a reference to one.  (Knowable only by looking at the code to
> shelve, since the syntax hides it, right?)

Erk?  I've never looked at the code to shelve, so, no, you don't
need to look at that code.

The code
	container[key].mutate()
evaluates container[key] (which might mean calling the container's
__getitem__ method), obtaining thereby an object, and calls the
given method of that object.  This is the case for dicts or
anything subscriptable, including shelves.

In the case of dicts, container[key] evaluates to a reference to
the very object which is in the dict, so mutating the value of
this expression mutates the object in the dict.  In the case of
shelves, container[key] evaluates to a reference to an object
which is the result of unpickling the contents of a record found
in the backing dbm database; that object isn't the same thing as
the record in the database, so mutating it doesn't alter that
record.

No knowledge of the internals of shelve is necessary.  It's just
expression evaluation plus the concept of shelve as a combination
of a key-value database on disk (dbm) and a generic object
serialization protocol (pickle).

Am I skipping over some important assumption without realizing it?

> > You want *every* change to the object to cause the shelve to be
> > updated immediately?  Really?
> 
> I certainly want every change to happen to the in-memory copy, which
> was what alerted me to the problem; having a sync() to flush it out to
> disk would suffice.  [...]

The change *is* happening to an in-memory copy when you do
	container[key].mutate()
It's just that the mutated copy is immediately discarded and not
written to the shelf.

> [...] But yes, in this particular case I do want live
> changes.  If it were large, using a real database might make sense,
> but only *after* benchmarking shelve and finding it slow, which I
> don't expect to :-)

*chuckle*  Good point.  (I assume you also don't need multiple
users reading and writing simultaneously.)

> (Note that I'm coming from perl, and would have done this with a tie'd
> hash there.)

I don't know Perl, as it happens.  What's a tied hash?

> > That's pretty hard in general, since the shelve-like thing can't
> > know a priori what operations mutate an object and what operations
> > don't.  (It could make an effort to detect changes after the fact
> 
> Or just pass back wrapped-objects, whose methods are "like" the input
> type but commit themselves on __setitem__ calls?

In the specific case of shelved lists and dicts, yes, that's a
good start.  We'd have to trap __setslice__ calls and others as
well, but I'll agree that it's doable.

Now, what about shelved foobar.wombat objects?  Which of their
methods should cause commits?

  [...]
-- 
Steven Taschuk             "The world will end if you get this wrong."
staschuk at telusplanet.net     -- "Typesetting Mathematics -- User's Guide",
                                 Brian Kernighan and Lorrinda Cherry