[Borgbackup] Any way to make introspection of backups more performant?

[Helge Milde]

I've been experimenting with moving from rdiff-backup to borgbackup. Mostly I'm in favor of switching (creating backups is a lot faster, it uses less disk space, allows for encryption, `borg mount` is an excellent tool, less chance of the backup becomming corrupt, allows for monthly/yearly backups, in active development etc..). The server I've been testing on is an e-mail server using ~2.7TB of disk space currently (so lots of small files).

Extraction of data after a `borg mount` is very fast. The `borg mount` itself is very slow (8 minutes), and similarly `borg extract` is slower than the equivalent rdiff-backup command, but there seems to be a static time where borg builds an internal structure of the backup before the extraction actually happens. So rdiff-backup would be faster for extracting smaller amounts of data, but extracting a full backup seems like it will be a lot faster with borg, which is more important.

The main problem I have is that gleaning any information from a backup (e.g. `borg list` or using `borg mount`) takes a lot of initial time. Whereas with rdiff-backup I can do `ls rdiff-backup/increments/var/www.*` to find the full history of a certain directory in the backup immediately (every creation, deletion or modification). This makes it possible to find exactly when the customer deleted their emails, or exactly when a site was hacked, and thus enable me to restore the most recent good backup of it pretty quickly. Is there anything that can enable this type of introspection of the full backup repository with borg?

I guess this is just a design difference, in which case you can consider this a feature request :). But perhaps there's still something I can tweak (borg init/create commands, configuration, or give the backup server more than 8GB RAM) to make borg mount/list faster. I've realised I could save the output of the `borg create --filter=AME` commands as log files, and create tools that enables me to do similar tasks.

On my backup repository, the `borg init` and the first `borg create` was done with borg 1.0.9-1~pbo8+1 (from jessie-backports). After that I changed to borg 1.1.7 from Github releases and did 3 more backups. I've read somewhere about borg having some bad defaults on earlier versions (something about chunk size?), though that might be talking about very old versions, so not sure if recreating the repository has any effect on performance.

Here's some information about the repository & timing of some borg commands: http://dpaste.com/3DXGN4A