survey: is shelve broken? should it be fixed?

[Steve Holden]

"Alex Martelli" <aleax at aleax.it> wrote ...
> Just checking if I'm the only one to feel that shelve seriosly violates
the
> principle of least astonishment when you shelve modifiable values:
>
> [alex at lancelot Lib]$ python
> Python 2.2.1 (#1, Apr 15 2002, 17:55:14)
> [GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import shelve
> >>> s=shelve.open('ciao','c')
> >>> s['ciao']=range(4)
> >>> s['ciao']
> [0, 1, 2, 3]
> >>> s['ciao'].append(23)
> >>> s['ciao']
> [0, 1, 2, 3]
>
> The access to s['ciao'] returns a temporary and promptly forgets about
> it.  The .append call is done on the temporary object and thus 'silently
> ignored' from the point of view of the shelf.  I must have helped people
> out of this trap a dozen times (plus, fallen into it myself two or three
> times:-) -- Python so rarely violates the principle of least astonishment
> that it's *really* astonishing when it does:-).
>
> We probably can't change this default (mis-)behavior due to backward
> compatibility needs, but it wouldn't be too hard to at least give a simple
> _optional_ way out, even though off-by-default:
>
> >>> s.close()
> >>> s=shelve.open('ciao','c',smart=1)
> >>> s['ciao']
> [0, 1, 2, 3]
> >>> s['ciao'].append(23)
> >>> s['ciao']
> [0, 1, 2, 3, 23]
> >>> s.close()
> >>> s=shelve.open('ciao','c')
> >>> s['ciao']
> [0, 1, 2, 3, 23]
> >>>
>
>
http://sourceforge.net/tracker/index.php?func=detail&aid=553171&group_id=547
0&atid=305470
>
> is a Q&D fix enabling this optional smart=1 parameter (doing it the RIGHT
> way would no doubt involve weak references, but I didn't try that yet --
it
> takes a bit more care because the values may or may not be weakly
> referenceable etc, I'd guess).
>
> But, implementation issues apart, what do people think of the general
issue
> of functionality and interface to it?  Am I the only silly Pythonista who
> worries about this, and everybody else once they've learned the trap just
> deftly avoids shelving mutables or uses other workarounds (or eschews
shelve
> altogether I guess:-)...?
>
> I think that having the 'smart=1' option in the docs would help people be
> _aware_ that the trap exists, and thus would be beneficial anyway.  Maybe
> we should just document the trap, but, as fixing it (at least optionally)
> isn't that hard, that doesn't strike me as ideal.
>
> Opinions welcome -- that's why I called this a 'survey'.  Thanks!

It certainly seems like the default behavior is broken, in a way that some
people may unfortunately have written code to depend on.

It seems like a good idea to go ahead with the modification, and (sadly) to
retain the existing behavior for compatibilty and absence of code breakage.

I'd also suggest the new behavior becomes the default for Python 3 ...

regards
 Steve
--

Steve Holden: http://www.holdenweb.com/ ; Python Web Programming:
http://pydish.holdenweb.com/pwp/