[Borgbackup] A small compression test

[Boris Kirkorowicz]

Hello,

Am 07.03.23 um 09:31 schrieb Thorsten Schöning:
> Guten Tag Bzzzz,
> am Freitag, 3. März 2023 um 00:21 schrieben Sie:
> 
>> So, if this ML accepts attachments (?) and that may help somebody to
>> make a choice, here they are.
> 
> Hi everyone,
> 
> thanks for sharing these numbers, found them pretty interesting and
> made me reconsider which compression mode to use. Tried with "zstd,3"
> now and see somewhat strange results with my two use-cases of backing
> up many infividual files of within a host vs. few large files of
> VM-images and database dumps.

is it possible to change the compression mode using the same (large) repo?
And, if it is possible, does it make sense?


> According to your numbers I would have expected increased backup times
> of about factor 4, but for some hosts I see factor 16! What makes me
> especially wonder is that things seem to scale with the number of
> files being backed up and not with the amount of actual data to store.
> The latter is MUCH more for my VM-images and database dumps, but their
> processing times mostly didn't increase at all.
> 
>  From my understanding, compression times shouldn't depend on the
> number of files or stuff, but only on the amount of data to be
> compressed and how good it can be compressed, shouldn't it?

I am not sure if I understand you right. That's what I see here:
I'm backing up a large amount of data, that will take several weeks. So, 
I interrupt the nightly backup every morning via kill command, so borg 
runs about 6 hours every night. On some days, it is possible to let borg 
run longer, say for 20 hours. While during normal nights (~6h) the 
throughput is about 11~14 GB/h compressed data, during the longer 
sessions (~20h) reaches 19~20 GB/h. That means e.g. 73 GB in 5:45h vs. 
421 GB in 21:55h.



-- 
Mit freundlichem Gruß                                 Best regards
                                  Kirkorowicz