[Borgbackup] Windows troubles

Sat Jun 10 07:54:04 EDT 2017

Hi,

I'm having some troubles with backups made from a remote windows box. Before submitting any bugs, I would be glad to have some feedback on what is expected/normal.

First, my setup : I installed borg on windows 10 using billyc's installer (https://github.com/borgbackup/borg/issues/440#issuecomment-279491415)(so, cygwin), and borg is backing up on a remote linux box which also have borg.
As the volume of data was huge, I first made two local backups from the linux box, using a usb drive. These backups seems fine.
Later, I made several (~10) remote backup from the windows box, some of them being partial (.checkpoint):
2017-04-09                           Sun, 2017-04-09 18:31:07 <- local
2017-04-12                           Wed, 2017-04-12 20:02:29 <- local
2017-05-30.checkpoint                Tue, 2017-05-30 20:39:30 <- remote
2017-06-01.checkpoint                Thu, 2017-06-01 01:00:25 <- remote
2017-06-01.checkpoint.1              Thu, 2017-06-01 20:53:24 <- remote
2017-06-01.checkpoint.2              Thu, 2017-06-01 21:06:14 <- remote
2017-06-04                           Sun, 2017-06-04 21:28:42 <- remote
2017-06-06                           Tue, 2017-06-06 01:00:26 <- remote
2017-06-07                           Wed, 2017-06-07 01:00:26 <- remote
2017-06-09                           Fri, 2017-06-09 01:00:24 <- remote
2017-06-10                           Sat, 2017-06-10 01:00:28 <- remote

The two "local" backups and the last remote one (2017-06-10) have a original size of ~860GB
Each of the "remote" backups had a dedup. size of 500-1000 MB, and a original size of ~940GB, due to the inclusion of a non-needed folder (80GB containing various globally unique data, which would, I guess, be at least 50GB after dedup+compression).

1) I deleted all but the last "remote" backups (wanting to reclaim the ~80GB lost). Before deletion:
  >borg info backupBorg::2017-06-10                                                                                                                                                                                 
  Name: 2017-06-10
  Fingerprint: d79c57fada25ab7d3e6bb07a0e642d7d28ffb704c333a56d814490c991b232d7
  Hostname: UN
  Username: ****
  Time (start): Sat, 2017-06-10 01:00:28
  Time (end):   Sat, 2017-06-10 01:17:23
  Command line: /usr/bin/borg create -sv -C lzma,5 --exclude-from borg//backupExclude ssh://****@****///backupBorg::{now:%Y-%m-%d} /BackupSource/
  Number of files: 317235
                         Original size      Compressed size    Deduplicated size
  This archive:              859.60 GB            663.66 GB            736.28 MB
  All archives:                7.73 TB              6.07 TB            557.82 GB
                         Unique chunks         Total chunks
  Chunk index:                  538363              5645695

And after deletion (omitting identical first lines):
  >borg info backupBorg::2017-06-10
                         Original size      Compressed size    Deduplicated size
  This archive:              859.60 GB            663.66 GB             77.26 GB
  All archives:                2.55 TB              1.99 TB            554.58 GB
                         Unique chunks         Total chunks
  Chunk index:                  532222              1851377

Having reclaimed only 3GB seemed suspicious, so I ran a repair :
  >borg check --repair backupBorg                           
  132 orphaned objects found!
  Archive consistency check complete, problems found.

After that :
  >borg info backupBorg::2017-06-10
                         Original size      Compressed size    Deduplicated size
  This archive:              859.60 GB            663.66 GB             77.26 GB
  All archives:                2.55 TB              1.99 TB            554.63 GB
                         Unique chunks         Total chunks
  Chunk index:                  531596              1850784

How can the All archive Dedup size be bigger !?
And how come I did not recover the ~50-80GB from the non-needed folder ?
Should I worry about the health of my repo ?

2) This might be related, but mounting the "remote" backups end up in some folders (but not all) being unreadable.
ls throws : 
  ls: error reading directory .: Input/output error (original in french: "ls: lecture du répertoire .: Erreur d'entrée/sortie")
while borg throws (having used borg mount -f) : 
  Exception after kill:
  Traceback (most recent call last):
    File "handlers.pxi", line 433, in llfuse.capi.fuse_releasedir (src/llfuse/capi_linux.c:11578)
    File "handlers.pxi", line 434, in llfuse.capi.fuse_releasedir (src/llfuse/capi_linux.c:11533)
    File "/usr/lib/python3/dist-packages/llfuse/pyapi.py", line 462, in releasedir
      def releasedir(self, fh):
    File "/usr/lib/python3/dist-packages/borg/helpers.py", line 1290, in handler
      raise exc_cls
  borg.helpers.SigTerm
  Exception after kill:
  Traceback (most recent call last):
    File "handlers.pxi", line 406, in llfuse.capi.fuse_readdir (src/llfuse/capi_linux.c:11058)
    File "handlers.pxi", line 411, in llfuse.capi.fuse_readdir (src/llfuse/capi_linux.c:10962)
    File "misc.pxi", line 32, in llfuse.capi.fill_c_stat (src/llfuse/capi_linux.c:16219)
  OverflowError: can't convert negative value to uid_t
  (last 6 lines repeated 2 times)

However, borg check does not return any error, and so did a borg extract --dry-run. I also hand-extracted some files (3-4), and everything is matching the original.
Anything I might try to help devs investigate, or is windows considered "not supported" ?
Again, should I worry about the health of my repo ?

Thanks a bunch,

N