[Borgbackup] faster / better deletion, for a bounty?

[Thomas Waldmann]

>>> (3) borg delete was incredibly slow for me. I killed it after two hours,
>>>     and it had read 500GB of the archive by then (reported with iotop).
>>>     I understood from IRC discussion that both prune and delete would
>>>     require reading the full 3.4 TB once per run, to sanitize some index?
>> No, they usually do not need to read all your data.
>>
>> The worst case might be that, though.
> 
> Can you elaborate on this?

Well, I think it is very rare / synthetic, but one could imagine an
archive referencing one chunk in every segment file.

If at time of deletion of that archive these references are the last
references to these chunks, the chunks get unused.

borg 1.0 behaviour is to compact segments to free up space
(unconditionally, iirc).

So, if there is a unused chunk in every segment (unused, but allocated
disk space), it would read all the still used chunks from these segments
and create new, compact segments from them.

borg 1.1 introduces a threshold, so it won't compact segments if there
is only little gain.

> BTW, thanks to you and everyone for all the work you've done on
> Borgbackup.  It has made incredible progress since I last looked at
> Attic (which was right around the time of the Borg fork.)  I'm seriously
> evaluating a switch from Obnam.

Great. :) Your blog post from 2015 always shows in top search results
when searching for a comparison of obnam, attic (borg), ...

Would be great to have a refresh of that at some time.



-- 

GPG ID: 9F88FB52FAF7B393
GPG FP: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393