[Borgbackup] purge not deleting data?

[devzero at web.de]

thanks for your answer/hints

> Well, guess you do not create a borg archive every day?

i do. cronjob runs every day, also on weekend.

first it does rsync to local, then borgbackup is run. rsync does delete data every 
day, there are servers with significant change every day (gb`s of added and deleted files)

i create a repository for every single server. that makes deduplication less efficient, yes, 
but deduplication benefit on the  "multiple servers into one borg repo) does not bring that great 
benefit like having each server in a dedicated borg repo.

dedup on per server basis is sufficient and more safe. what i want is compression and rotation/prune  
(i.e. incremental forever) and dedup for the data, which is very different on every server anyway...

we have the same backup strategy from another location , but with rsync + zfs, but i did not want
to rely on zfsonlinux as sole strategy for backup, so i created a second solution working similar, 
but with borgbackup as the backend instead of zfs + rotating snapshots

> A while ago I tried to explain prune policies better and made a
> graphical example, see there:
> 
> https://github.com/borgbackup/borg/blob/1.0-maint/docs/misc/prune-example.txt
> 
> BTW, I've seen you use d-m-y for dates, that is not the best way to do
> it as it does not sort correctly. Maybe rather use y-m-d, if possible.

i will see what i can find and thanks again for the hints.

regards
roland


> Gesendet: Montag, 23. Januar 2017 um 14:44 Uhr
> Von: "Thomas Waldmann" <tw at waldmann-edv.de>
> An: borgbackup at python.org
> Betreff: Re: [Borgbackup] purge not deleting data?
>
> Hi,
> 
> > every few days, i see that borg prune does not appear to purge data from the repos,
> > and for my curiosity it happens for all repos at the same day/backup-run
> 
> Well, guess you do not create a borg archive every day?
> 
> E.g. because your cronjob does not run sundays or so?
> 
> Or, due to other circumstances (e.g. machine powered off) it did not
> happen to make a backup every day?
> 
> It could also be just a not-so-obvious effect of implementing your prune
> policy, like the different parts (daily, weekly, monthly, ...) interfering.
> 
> > - though rsync tells it did delete files from the rsync copy.
> 
> That is completely unrelated, it just reduces total size and count of
> files in your backup data source.
> 
> > so i wonder under which circumstances borg prune skips deleting files !?
> 
> borg prune deletes ("thins out") backup archives following the policy
> you give to it. It just decides which backup archives to keep / not to
> keep, so the policy you gave is implemented. If you give -v --list, it
> will spill out the result of that decision.
> 
> As a consequence of archives getting deleted (if your repo is not
> append-only) it will also usually reduce your overall repo size IF it
> decided to delete at least 1 old archive.
> 
> > as the repos were created at different points in time i cannot explain this to me, maybe someone has a clue how this can be explained or analyzed?
> 
> The effects of non-trivial purge policies (maybe combined with not
> making a backup every day) are often a bit hard to explain.
> 
> A while ago I tried to explain prune policies better and made a
> graphical example, see there:
> 
> https://github.com/borgbackup/borg/blob/1.0-maint/docs/misc/prune-example.txt
> 
> BTW, I've seen you use d-m-y for dates, that is not the best way to do
> it as it does not sort correctly. Maybe rather use y-m-d, if possible.
> 
> Cheers, Thomas
> 
> _______________________________________________
> Borgbackup mailing list
> Borgbackup at python.org
> https://mail.python.org/mailman/listinfo/borgbackup
>