[Borgbackup] Question on "borg mount" performance

[public at enkore.de]

sha256sum does 32k reads on my system; cp reads 131k blocks. I'm not
sure, but at those sizes I wouldn't expect FUSE to aggregate reads to
reduce call overhead.

For a large file I would expect to be significantly slower than "borg
extract", since it would decrypt/decompress a 2 MB chunk 63 times for
32k reads, and still 15 times for 131 kB reads, assuming those arrive at
Borg with that sizing.

That being said performance might be improved here by having a LRUCache
of partially read chunks.

Ref: https://github.com/borgbackup/borg/blob/master/borg/fuse.py#L224

Cheers, Marian

On 22.04.2016 19:09, Thomas Waldmann wrote:
>> I was just now going to do some verification of of the backups, by
>> comparing the SHA1 sum of the original VM disk image ( a clone, really,
>> so it doesn't chnage) to the SHA1 sum of the backup. I first did a "borg
>> mount", and did "sha1sum /mnt/.../disk.image" directly from the FUSE
>> mount. This was horribly slow. I then did a "borg extract" of that same
>> disk image, and that was much, much faster.
>>
>> Is this what you expected?
>
> Well, it is expected that FUSE mount is slower than extract.
>
> You didn't write how slow it was so that we could directly compare.
>
> Also the operations you did were slightly different:
>
> a) your ran sha1sum directly on the file in the fuse mount - that might
> behave differently than copying the file from the fuse mount to
> somewhere else with cp depending on how many and how big read calls
> sha1sum does.
>
> The FUSE mount also needs to do some preprocessing when you first enter
> a "archive directory" so it can know what file names it shall show you
> there.
>
> It also does some caching to avoid having to get same chunks multiple
> times from the repo
>
> b) extract doesn't do most of that
>
> In general, if you know that you will deal with a huge amount of data
> (like a full extract, like extracting gigabytes), borg extract is more
> efficient.
>
>> In any case, Borg is wonderfully documented, very complete in the
>> features I need, and is written in Python. What could be better?
>
> :)
>