[Borgbackup] Backup taking very long - sometimes

[Samuel Greiner]

Thank you for your advices,

I'm now using the most recent version 1.1.13.

Still the problem persists.

With the option "--list --filter="AME"" I see that some backups run very
fast and only a considerable amount of files is changed or added.
The backups that have a really long duration backup a whole lot of files
as modified which should be untouched.

The source is a nextcloud data directory and the data is on a btrfs volume.

Do you have advice how I could investigate this further?

Thank you very much
Samuel



Am 10.09.20 um 10:47 schrieb Winfried Schürmann:
> Just a hint:
> it's easy to install the binaries of Borg 1.1.13 not waiting for updates
> of repos:
> https://borgbackup.readthedocs.io/en/stable/installation.html#pyinstaller-binary
>
> Winfried
>
>
> Am 10.09.20 um 09:52 schrieb Samuel:
>> Okay.
>>
>> I'm now using 1.1.11 from buster-backports.
>>
>> Next time I will try to shorten the logs to the relevant part.
>>
>> You are right, there is a huge amount of files classified as modified,
>> which should be unchanged (for several years).
>> Now I'm unsure in how to investigate further.
>> The files are stored on a btrfs filesystem across 4 HDs as RAID1.
>>
>> Unfortunately I didn't find a hint in the FAQ regarding this behaviour.
>>
>> I will now wait for the next Backups with the more recent version of
>> borgbackup and try to investigate the new logs.
>>
>>
>> Thanks a lot for your help!
>> Samuel
>>
>> Am 09.09.20 um 12:34 schrieb Thomas Waldmann:
>>>> Stats was already enabled and the duration of the backup doesn't seem to
>>>> depend on the data added (i attached the of the borg backup).
>>> You should use a more recent release than borg 1.1.9, see the advisory
>>> at the top of the changelog.
>>>
>>>> Logs with stats enabled but without --list --filter="AME" can be found here:
>>>> https://pastebin.com/raw/RW43kBHi
>>> Next time you could shorten the logs so they only show the relevant /
>>> interesting stuff.
>>>
>>> Didn't see anything remarkable there (except maybe that your "System"
>>> backup includes a rather high count of files).
>>>
>>>> With --list --filter="AME" enabled the logs since 2020-09-06 exceed
>>>> 200Mbyte. Therefore i can't send them.
>>> It wasn't intended that you publish that log, but rather that you check
>>> yourself whether the A)dded and M)odified files shown in there are
>>> somehow as expected.
>>>
>>> If it shows files as M)odified that had no content modification, that is
>>> an indication that the borg files cache maybe doesn't work ok for you /
>>> for that filesystem type or that the files were slightly "touched" by
>>> something.
>>>
>>> You have to find out then what the root cause for this is, see the FAQ
>>> and the documentation of --files-cache.
>>>
>>> Or if it shows a huge amount of E)rrors, then find out if the errors are
>>> severe or if they can be ignored (sometimes one can just exclude
>>> problematic stuff if it does not need to be backed up anyway).
>>>
>>>
>>>
>>>
>>>> Am 04.09.20 um 14:29 schrieb Thomas Waldmann:
>>>>>> Borg runs on a regular daily basis and is working stable.
>>>>>> But the consumed cpu time varies between around 4 hours and 12-13 hours:
>>>>> Could be because on different days, a different amount of data changes?
>>>>>
>>>>> Or (if you do backups to a remote repo server), the connection speed to
>>>>> / system load of source or target server is different.
>>>>>
>>>>> Without more infos, it is hard to say.
>>>>>
>>>>> You could start by adding this to borg create:
>>>>>
>>>>> --stats --list --filter="AME"
>>>>>
>>>>> stats should tell if there was really much data added.
>>>>> if so, the list output should hint about what it was.
>> _______________________________________________
>> Borgbackup mailing list
>> Borgbackup at python.org
>> https://mail.python.org/mailman/listinfo/borgbackup
>>
> _______________________________________________
> Borgbackup mailing list
> Borgbackup at python.org
> https://mail.python.org/mailman/listinfo/borgbackup