[Borgbackup] 550GB rsync-tree will not dump

[Thomas Waldmann]

>> What is the error when you use no compression?
> 553.53 GB O 553.75 GB C 319.15 GB D 4585967 N

OK, so 550GB data and 4.5M files.

>   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line
> 704, in _read
>     segment, offset))
> borg.helpers.IntegrityError: Invalid segment entry size [segment 48441,
> offset 4210392]

It has written a huge object and now rejects to read it, see:

https://github.com/borgbackup/borg/issues/1451


What we are wondering about:

You don't have that many files, it is just 4.5 million files.
Also they are not that big as the total size is just 550GB.

Also, you said you have a lot of hardlinks (which borg stores as
chunkless item that keeps a reference to a "master item" with the chunks
list - but it will store ACLs and xattrs for the hardlink item separately).

So why are your metadata items that big? Do you have a lot of ACLs or
xattrs?

See there:

https://github.com/borgbackup/borg/issues/1452

-- 

GPG ID: 9F88FB52FAF7B393
GPG FP: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393