[Borgbackup] Decreasing speed of borg check command?

[Marian Beermann]

Repository check _should be_ I/O bound unless the CPU is very slow.

Archive check should generally be latency / CPU bound.

-Marian

Am 05.04.19 um 12:37 schrieb Roland @web.de:
> i think "borg check" is cpu bound, not io bound.
> 
> so, have a look at "top" - you should see borg process at 100% cpu for
> most of the time.
> 
> why amount of read from disk varies is a good question.... but at least
> i do see
> 
> cpu at 100% most of the time and memory consumption is constant, too
> 
> 
> Am 04.04.19 um 23:22 schrieb Marcin Zajączkowski:
>> Hi,
>>
>> I would like to consult one observation. Performing a 600GB+ repo check,
>> the HDD read speed started at ~110MB/s (USB3 HDD drive), but over time
>> tends to decease up until ~60MB/s after 2 hours. Disk transfer data from
>> dstat.
>>
>>> ----system---- --dsk/sdc-- -net/total- ---system-- ----total-usage----
>>>       time     | read  writ| recv  send| int   csw |usr sys idl wai stl
>> ...
>>> 02-04 23:28:20| 111M    0 | 213   269 |2620  5023 |  6   7  84   3   0
>>> 02-04 23:28:25| 110M    0 | 171   199 |4277  8007 | 13   8  72   7   0
>>> 02-04 23:28:30| 110M    0 | 342   199 |3926  8305 | 12   7  73   6   0
>> ...
>>> 03-04 00:03:50|  95M    0 |   0     0 |2827  5536 |  7   6  81   5   0
>>> 03-04 00:03:55|  93M    0 |   0     0 |4306    14k|  9   7  80   4   0
>>> 03-04 00:04:00|  95M    0 |   0     0 |3366  6121 |  8   7  79   6   0
>> ...
>>> 03-04 01:18:20|  57M    0 |   0     0 |1294  2557 |  3   4  91   2   0
>>> 03-04 01:18:25|  57M    0 |   0     0 |1184  2433 |  2   4  90   3   0
>>> 03-04 01:18:30|  57M    0 | 592   592 |1301  2339 |  3   4  92   1   0
>>
>> It may caused by some other reasons (but rather not a fragmentation - it
>> was after an initial test backup performed to an empty disk), but it
>> could be also some kind of memory leak in Borg which leads to keeping
>> more and more data internally over time causing that slowness.
>>
>> Do you have an idea what could it be caused by?
>>
>> I may try reproduce that behavior later on this month if needed. Borg
>> 1.1.8, Fedora 29.
>>
>> Marcin
>>
> _______________________________________________
> Borgbackup mailing list
> Borgbackup at python.org
> https://mail.python.org/mailman/listinfo/borgbackup