pickle->zlib->shelve

[Jeff Bauer]

Thomas Weholt wrote:
> I was thinking maybe I could use pickle and compress the pickled object
> too, before storing it into the database and save some space. Pickled
> objects also seem to have lots of repetitive data.
> 
> Now my question is :
> 
> 1. has this been tested and found ineffective in relation to a) speed of
> decompression or b) size of objects not big enough to gain any
> significant size advantage by compression
> 2. if not tested, why? Maybe a subclass of pickle could implement
> compression and decompression of objects?

Thomas,

I would hazard a guess that the reason we haven't seen
compression built into the pickler:

a) nobody's been motivated yet (go to it! ;-)
b) potentially non-portable (not all Python environments
have access to zlib)

I don't see any reason why you shouldn't be able to
compress pickles intended for your own use.

Best regards,

Jeff Bauer
Rubicon Research