[Borgbackup] Backup taking very long - sometimes

[Samuel]

Okay.

I'm now using 1.1.11 from buster-backports.

Next time I will try to shorten the logs to the relevant part.

You are right, there is a huge amount of files classified as modified,
which should be unchanged (for several years).
Now I'm unsure in how to investigate further.
The files are stored on a btrfs filesystem across 4 HDs as RAID1.

Unfortunately I didn't find a hint in the FAQ regarding this behaviour.

I will now wait for the next Backups with the more recent version of
borgbackup and try to investigate the new logs.


Thanks a lot for your help!
Samuel

Am 09.09.20 um 12:34 schrieb Thomas Waldmann:
>> Stats was already enabled and the duration of the backup doesn't seem to
>> depend on the data added (i attached the of the borg backup).
> You should use a more recent release than borg 1.1.9, see the advisory
> at the top of the changelog.
>
>> Logs with stats enabled but without --list --filter="AME" can be found here:
>> https://pastebin.com/raw/RW43kBHi
> Next time you could shorten the logs so they only show the relevant /
> interesting stuff.
>
> Didn't see anything remarkable there (except maybe that your "System"
> backup includes a rather high count of files).
>
>> With --list --filter="AME" enabled the logs since 2020-09-06 exceed
>> 200Mbyte. Therefore i can't send them.
> It wasn't intended that you publish that log, but rather that you check
> yourself whether the A)dded and M)odified files shown in there are
> somehow as expected.
>
> If it shows files as M)odified that had no content modification, that is
> an indication that the borg files cache maybe doesn't work ok for you /
> for that filesystem type or that the files were slightly "touched" by
> something.
>
> You have to find out then what the root cause for this is, see the FAQ
> and the documentation of --files-cache.
>
> Or if it shows a huge amount of E)rrors, then find out if the errors are
> severe or if they can be ignored (sometimes one can just exclude
> problematic stuff if it does not need to be backed up anyway).
>
>
>
>
>> Am 04.09.20 um 14:29 schrieb Thomas Waldmann:
>>>> Borg runs on a regular daily basis and is working stable.
>>>> But the consumed cpu time varies between around 4 hours and 12-13 hours:
>>> Could be because on different days, a different amount of data changes?
>>>
>>> Or (if you do backups to a remote repo server), the connection speed to
>>> / system load of source or target server is different.
>>>
>>> Without more infos, it is hard to say.
>>>
>>> You could start by adding this to borg create:
>>>
>>> --stats --list --filter="AME"
>>>
>>> stats should tell if there was really much data added.
>>> if so, the list output should hint about what it was.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/borgbackup/attachments/20200910/27740514/attachment.html>