[Borgbackup] Baffling behavior on low cpu system with borg backup

[tmhikaru at gmail.com]

	Been a while. Sorry about this, I injured my wrist and it is hard to
type one handed.

	Did tests over the weekend. Despite everything I tried, I could not
coerce the RPI into syncing properly, so out of frustration, I tried doing
this to see if it'd do something differently:
https://github.com/borgbackup/borg/pull/238

	To my surprise, it started working and synced, and did a full backup
without any trouble!  Immediately afterwards I modified the manifest line in
the repo config as you had suggested and ran the test backup again, and it
synced and correctly id'd most files as unmodified.  I recorded memory use
for both and used time correctly this time, and will attach both as gzips to
this message.  Here is the cache data you wanted:

RPI, 32bit armv6 ~480MB of ram:
root at raspberrypi:~# ls -l .cache/borg/40f5689bc9c5faac015dd94283a4dfb42dc5361bf0c77fd1402c365235ebd8f9/chunks
-rw------- 1 root root 96336214 Jun 24 21:17 .cache/borg/40f5689bc9c5faac015dd94283a4dfb42dc5361bf0c77fd1402c365235ebd8f9/chunks

64Bit x86 2xXeon 18GB of ram running fedora 23:
[root at roll ~]# ls -l .cache/borg/40f5689bc9c5faac015dd94283a4dfb42dc5361bf0c77fd1402c365235ebd8f9/chunks
-rw-------. 1 root root 96336214 Jun 26 00:52 .cache/borg/40f5689bc9c5faac015dd94283a4dfb42dc5361bf0c77fd1402c365235ebd8f9/chunks


Now, the interesting thing is, this was not a permanent fix! Despite that it
was able to sync twice in a row, a few days later when I tried to run a full
backup using my script which is run the same way my test was, it got
completely jammed and was still trying to sync one of the archives it had
worked through days before in less than an hour in both previous tests, but
six hours later when I came home it was just sitting there hammering away at
the cpu.  I have since given up on trying to use it this way, and am back to
using sshfs.  sshfs is much faster despite its drawbacks since there is no
cache to sync, and apparently borg client reading sshfs on the server doing
a full backup runs ~1 hr faster than it does running on the RPI even not
counting sync time.  I will read up on xattrs some more, maybe there is a
solution I can use to store them without having to rely on borg to do it
directly.

I have NOT moved, removed or altered the cachedir since I
killed the borg client on the RPI so if anything useful may be in there to
find out why it is getting hung up, tell me and I'll pull it out.


To be 100% clear, both of the attached tests completely worked and did not
freeze up.  The first one did a full backup of everything on the RPI to the
remote server, and the second one just synced and then skipped 99% of
everything on the RPI since it hadn't been modified.

I am still utterly clueless as to why this is happening. Ideas?

Tim McGrath
-------------- next part --------------
A non-text attachment was scrubbed...
Name: borglog.first.gz
Type: application/gzip
Size: 3397 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/borgbackup/attachments/20160628/17cb0f1f/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: borglog.second.gz
Type: application/gzip
Size: 2182 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/borgbackup/attachments/20160628/17cb0f1f/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/borgbackup/attachments/20160628/17cb0f1f/attachment.sig>