[Borgbackup] Central backup and ~/.cache/borg/ piling up

[William Kenworthy]

Could you offload the long term backups onto a machine with more
resources? - That is:

1. create your normal backups to a machine with plenty of resources

2. Back those borgbackup repositories up to a single local repo on the
backup machine (i.e., backup the backups)

3. heavily prune the first stage backups to regain the space.

The heavily pruned first stage only needs one or two iterations to be
present so the local machines can reduce their cache but still be efficient

Restoration means you will need to restore the correct first stage
version from the applicable 2nd stage so you can get at the original files.

I have found that there is a lot of duplication between backups so the
2nd stage is usually quite fast - unless there are a lot of changes.  I
have had to extract two sets so far without problems, but there are the
extra steps/effort and time required. I my case, the first stage is on a
moosefs data store and the second stage is on an off-line removable hard
drive.

BillK


On 30/8/20 8:52 pm, Hans-Peter Jansen wrote:
> Am Sonntag, 30. August 2020, 14:01:58 CEST schrieb Thomas Waldmann:
>>>> The chunks.archive.d cache is O(archive count * archive size) big, so if
>>>> that is getting too big, just reduce the archive count via "borg prune".
>> Check whether the chunks.archive.d directory is eating your disk space.
> It is:
> $ du -h . chunks.archive.d/
> 17G     ./chunks.archive.d
> 21G     .
>
>> If so, you either can reduce archive amount or apply the hack from the
>> FAQ to get rid of chunks.archive.d (the latter might be ok, if only one
>> borg client will write to the repo, so cache rebuilds will never or
>> rarely happen OR if you are willing to take the time for the rebuilds).
> Will dive into this next.
>  
>>> discussion of --no-files-cache
>> Will make your backups slow as it will read/chunk/hash/lookup all files
>> content.
> Thanks,
> Pete
>
>
> _______________________________________________
> Borgbackup mailing list
> Borgbackup at python.org
> https://mail.python.org/mailman/listinfo/borgbackup