[Borgbackup] purge not deleting data?
Fabio Pedretti
fabio.pedretti at unibs.it
Wed Jan 25 12:09:37 EST 2017
Hi, I suggest using prune before create, so that prune always start at the
same time and is not impacted by the time the create requires. This way
you'll get consistent archive deletion. Another issue that may alter timing
of prune is this issue when older archivers cross a daylight saving time
change: https://github.com/borgbackup/borg/issues/1980
2017-01-23 12:39 GMT+01:00 <devzero at web.de>:
> Hello,
>
> i run a rsync based daily backup where a number of hosts files are being
> staged to some local filesystem (one subdir per host) and then put into
> individual borg-repo on a "one-dir-per-host" basis (see script below)
>
> every few days, i see that borg prune does not appear to purge data from
> the repos, and for my curiosity it happens for all repos at the same
> day/backup-run - though rsync tells it did delete files from the rsync copy.
>
> so i wonder under which circumstances borg prune skips deleting files !?
>
> as the repos were created at different points in time i cannot explain
> this to me, maybe someone has a clue how this can be explained or analyzed?
>
> regards
> roland
>
>
> [root at backupvm2]# egrep -Hi "This archive|Deleted" borg_*-*-2017*.err
> borg_04-01-2017_03-45.err:This archive: 60.42 GB
> 45.51 GB 536.55 MB
> borg_04-01-2017_03-45.err:Deleted data: -99.61 GB
> -82.57 GB -1.32 GB
> borg_05-01-2017_04-11.err:This archive: 60.39 GB
> 45.49 GB 1.17 GB
> borg_05-01-2017_04-11.err:Deleted data: -60.54 GB
> -45.61 GB -494.64 MB
> borg_06-01-2017_03-32.err:This archive: 60.37 GB
> 45.46 GB 1.12 GB
> borg_06-01-2017_03-32.err:Deleted data: -60.55 GB
> -45.62 GB -985.93 MB
> borg_07-01-2017_05-00.err:This archive: 60.40 GB
> 45.58 GB 1.32 GB
> borg_07-01-2017_05-00.err:Deleted data: -60.55 GB
> -45.63 GB -499.40 MB
> borg_08-01-2017_03-10.err:This archive: 60.40 GB
> 45.58 GB 536.57 MB
> borg_08-01-2017_03-10.err:Deleted data: 0 B
> 0 B 0 B
> borg_09-01-2017_03-24.err:This archive: 60.39 GB
> 45.58 GB 589.20 MB
> borg_09-01-2017_03-24.err:Deleted data: -60.54 GB
> -45.63 GB -473.20 MB
> borg_10-01-2017_03-34.err:This archive: 60.39 GB
> 45.59 GB 1.11 GB
> borg_10-01-2017_03-34.err:Deleted data: -60.54 GB
> -45.63 GB -511.70 MB
> borg_11-01-2017_03-51.err:This archive: 60.46 GB
> 45.63 GB 910.10 MB
> borg_11-01-2017_03-51.err:Deleted data: -60.54 GB
> -45.63 GB -508.64 MB
> borg_12-01-2017_03-51.err:This archive: 60.46 GB
> 45.64 GB 1.33 GB
> borg_12-01-2017_03-51.err:Deleted data: -60.54 GB
> -45.63 GB -507.77 MB
> borg_13-01-2017_04-50.err:This archive: 60.55 GB
> 45.71 GB 1.50 GB
> borg_13-01-2017_04-50.err:Deleted data: -60.55 GB
> -45.63 GB -513.74 MB
> borg_14-01-2017_04-32.err:This archive: 60.56 GB
> 45.71 GB 926.04 MB
> borg_14-01-2017_04-32.err:Deleted data: 0 B
> 0 B 0 B
> borg_15-01-2017_03-14.err:This archive: 60.54 GB
> 45.71 GB 565.45 MB
> borg_15-01-2017_03-14.err:Deleted data: 0 B
> 0 B 0 B
> borg_16-01-2017_03-49.err:This archive: 60.54 GB
> 45.71 GB 529.96 MB
> borg_16-01-2017_03-49.err:Deleted data: -60.46 GB
> -45.56 GB -512.22 MB
> borg_17-01-2017_04-14.err:This archive: 60.56 GB
> 45.72 GB 1.57 GB
> borg_17-01-2017_04-14.err:Deleted data: -60.44 GB
> -45.53 GB -358.81 MB
>
> ls -1 */rsync.log |while read line;do echo $line $(grep -i deleting
> $line|wc -l);done
> 2017-01-03-2303/rsync.log 599
> 2017-01-04-2303/rsync.log 622
> 2017-01-05-2303/rsync.log 2525
> 2017-01-06-2303/rsync.log 711
> 2017-01-07-2303/rsync.log 6305
> 2017-01-08-2303/rsync.log 581
> 2017-01-09-2303/rsync.log 488
> 2017-01-10-2303/rsync.log 1605
> 2017-01-11-2303/rsync.log 738
> 2017-01-12-2303/rsync.log 1669
> 2017-01-13-2303/rsync.log 5394
> 2017-01-14-2303/rsync.log 5056
> 2017-01-15-2303/rsync.log 478
> 2017-01-16-2303/rsync.log 487
>
>
> [root at backupvm2 backup]# cat borg_backup_all.sh
> #!/bin/bash
>
> export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
> export BORG_RELOCATED_REPO_ACCESS_IS_OK=yes
>
> export BORG_CACHE_DIR=/backup/borg-cache
> export DATUM=$(date +"%d-%m-%Y_%H-%M")
> export ARCHIVENAME=archive-$DATUM
> export BORGBIN="/backup/bin/borg"
> export BASEPATH="/iscsi/lun1/borg-repos"
> export BASEPATHLOG="/iscsi/lun2/borg-logs"
> echo "hostname::archive-name
> orig-size compr-size dedup-size"
>
> for HOSTNAME in $(ls -1r /btrfspool/backup)
> do
> if [ ! -f /btrfspool/backup/$HOSTNAME/disabled ]
> then
> export REPOPATH="$BASEPATH/$HOSTNAME"
> export ARCHIVEPATH="$REPOPATH::$ARCHIVENAME"
> export LOG=$BASEPATHLOG/$HOSTNAME/borg_$DATUM.log
> export ERR=$BASEPATHLOG/$HOSTNAME/borg_$DATUM.err
>
> if [ ! -d $REPOPATH ]; then
> mkdir $REPOPATH
> mkdir $BASEPATHLOG/$HOSTNAME
> $BORGBIN init --encryption=none $REPOPATH
> fi
>
> cd /btrfspool/backup/$HOSTNAME/backup
> $BORGBIN create --filter=AME --info --list --stats --numeric-owner
> --compression lz4 $ARCHIVEPATH . >$LOG 2>$ERR
> echo $HOSTNAME::$ARCHIVENAME $($BORGBIN info $REPOPATH::$ARCHIVENAME
> |egrep "This archive"|cut -d ":" -f 2-)| /usr/bin/awk '{printf "%-60s %10s
> %2s %10s %2s %10s %2s\n",$1,$2,$3,$4,$5,$6,$7}'
> $BORGBIN prune --verbose --stats --keep-daily 14 --keep-weekly 8
> --keep-monthly 6 $REPOPATH >>$LOG 2>>$ERR
> fi
> done
> _______________________________________________
> Borgbackup mailing list
> Borgbackup at python.org
> https://mail.python.org/mailman/listinfo/borgbackup
>
--
ing. Pedretti Fabio
Responsabile U.O.C. "Reti e Sistemi"
http://www.unibs.it/organizzazione/amministrazione-centrale/servizio-servizi-ict/uoc-reti-e-sistemi
Università degli Studi di Brescia
Via Valotti, 9 - 25121 Brescia
E-mail: fabio.pedretti at unibs.it
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/borgbackup/attachments/20170125/4f8c1d61/attachment.html>
More information about the Borgbackup
mailing list