[Borgbackup] Speed issues compared with rsync

Catscrash catscrash at catscrash.de
Fri Apr 14 11:06:54 EDT 2017


Hi,

thanks for your answer,
>> and compression to none
> Not sure about this one, consider trying lz4 (little cpu load, less data
> to transfer/store).

That leads to 14 hours on a non-first backup run, I did try that already
before and ended up starting from scratch with uncompressed to try if
that's faste

> The fast-skip detection depends on file mtime, file size and inode
> number. Check if these are stable for unchanged files.
> [...]
> borg should be similarly fast for most files unchanged.
>
> For changed files, borg has a bit more to compute (plus optionally
> encryption and compression) than rsync but it will save you a lot of
> storage in some cases (like big files with little changes, like VM images).

In the end it's only 67,22MB that's new, so it does recognize the files
as unchanged:

Time (start): Fri, 2017-04-14 04:00:13
Time (end):   Fri, 2017-04-14 12:17:43
Duration: 8 hours 17 minutes 30.63 seconds
Number of files: 258614
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              335.34 GB            335.34 GB             67.22 MB
All archives:              670.68 GB            670.68 GB            304.49 GB

                       Unique chunks         Total chunks
Chunk index:                  311269               744901

The largest file system taking up the most space is this one

/dev/mapper/LVM-Bilder on /srv/Bilder type ext4 (rw,relatime,data=ordered)

The files are not touched or changed, mtime is for most several years
ago, file size doesn't change as well and if inode would change, rsync
would not be able to hardlink, since it's depending on inode as far as I
know.
Atime on the other hand is of course every night when the backup runs,
but that shouldn't be an issue, right?
> borg 1.0 can only use 1 of the cores.
> borg 1.2 is planned to have multithreading.

Okay sure, that might be an issue, but still the difference seems a bit
huge for it just to be due to the cores used, right?
> Find out why fast-skip does not work. Is something touching the mtimes?
> Do you use a filesystem that does not have stable inode numbers? For the
> last case, there is a --ignore-inodes switch.

How do I know if it's fast-skipping or not? mtime is not changed, ext4
should have stable inode numbers. Nevertheless, I'm trying with the
--ignore-inodes switch now and will let you know

Thanks for your time!


More information about the Borgbackup mailing list