key/value store optimized for disk storage

Paul Rubin no.email at nospam.invalid
Fri May 4 04:01:29 EDT 2012


Steve Howell <showell30 at yahoo.com> writes:
> Makes sense.  I believe I got that part correct:
>
>   https://github.com/showell/KeyValue/blob/master/salted_compressor.py

The API looks nice, but your compress method makes no sense.  Why do you
include s.prefix in s and then strip it off?  Why do you save the prefix
and salt in the instance, and have self.salt2 and s[len(self.salt):]
in the decompress?  You should be able to just get the incremental bit.

> I'm pretty sure this happens for free as long as the salt is large
> enough, but maybe I'm misunderstanding.

No I mean there is some fixed overhead (a few bytes) in the compressor
output, to identify it as such.  That's fine when the input and output
are both large, but when there's a huge number of small compressed
strings, it adds up.



More information about the Python-list mailing list