Most space-efficient way to store log entries

[Chris Angelico]

On Thu, Oct 29, 2015 at 9:30 AM, Marc Aymerich <glicerinu at gmail.com> wrote:
> I'm writting an application that saves historical state in a log file.
> I want to be really efficient in terms of used bytes.

Why, exactly?

By zipping the state, you make it utterly opaque. It'll require some
sort of tool to tease it apart before you can read anything. Much more
useful would be to have some sort of textual delimiter, followed by
the content - then when you come to read it, all you need is a text
viewer.

Disk space is not expensive. Even if you manage to cut your file by a
factor of four (75% compression, which is entirely possible if your
content is plain text, but far from guaranteed), that's maybe three
years of Moore's Law at most. You can get 3-4 terabytes of storage for
roughly $100-$200, depending on exactly where you buy it, which dollar
you're using, etc. How long will your program have to run to generate
that much data? If you can't do that in, say, two years, don't bother
compressing.

ChrisA