key/value store optimized for disk storage

[Steve Howell]

On May 4, 1:01 am, Paul Rubin <no.em... at nospam.invalid> wrote:
> Steve Howell <showel... at yahoo.com> writes:
> > Makes sense.  I believe I got that part correct:
>
> >  https://github.com/showell/KeyValue/blob/master/salted_compressor.py
>
> The API looks nice, but your compress method makes no sense.  Why do you
> include s.prefix in s and then strip it off?  Why do you save the prefix
> and salt in the instance, and have self.salt2 and s[len(self.salt):]
> in the decompress?  You should be able to just get the incremental bit.

This is fixed now.

https://github.com/showell/KeyValue/commit/1eb316d6e9e44a37ab4f3ca73fcaf4ec0e7f22b4#salted_compressor.py


> > I'm pretty sure this happens for free as long as the salt is large
> > enough, but maybe I'm misunderstanding.
>
> No I mean there is some fixed overhead (a few bytes) in the compressor
> output, to identify it as such.  That's fine when the input and output
> are both large, but when there's a huge number of small compressed
> strings, it adds up.

It it's in the header, wouldn't it be part of the output that comes
before Z_SYNC_FLUSH?