[Borgbackup] Fail to run

Bzzzz lazyvirus at gmx.com
Mon Aug 29 11:17:10 EDT 2022 

Debian bullseye
all libs install
python pkg msgpack not installed
===================================
Hi list,

After a fresh pip3 install borgbackup (v.1.2.2), it fails on a brand
new repo with these messages :

A /SDC-PRECONVERT.gpt
Local Exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
5159, in main exit_code = archiver.run(args)
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
5090, in run return set_ec(func(args))
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
183, in wrapper return method(self, args, repository=repository,
**kwargs) File
"/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line 675, in
do_create create_inner(archive, cache, fso) File
"/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line 608, in
create_inner self._rec_walk(path=path, parent_fd=parent_fd, name=name,
File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
820, in _rec_walk self._rec_walk( File
"/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line 786, in
_rec_walk status = self._process_any(path=path, parent_fd=parent_fd,
name=name, st=st, fso=fso, cache=cache, File
"/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line 688, in
_process_any return fso.process_file(path=path, parent_fd=parent_fd,
name=name, st=st, cache=cache) File
"/usr/local/lib/python3.9/dist-packages/borg/archive.py", line 1434, in
process_file self.process_file_chunks(item, cache, self.stats,
self.show_progress, backup_io_iter(self.chunker.chunkify(None, fd)))
File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
1261, in process_file_chunks item.chunks.append(chunk_processor(chunk))
File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
1249, in chunk_processor chunk_entry = cache.add_chunk(chunk_id, data,
stats, wait=False) File
"/usr/local/lib/python3.9/dist-packages/borg/cache.py", line 949, in
add_chunk data = self.key.encrypt(chunk) File
"/usr/local/lib/python3.9/dist-packages/borg/crypto/key.py", line 370,
in encrypt next_iv =
self.nonce_manager.ensure_reservation(self.cipher.next_iv(), File
"/usr/local/lib/python3.9/dist-packages/borg/crypto/nonces.py", line
79, in ensure_reservation repo_free_nonce = self.get_repo_free_nonce()
File "/usr/local/lib/python3.9/dist-packages/borg/crypto/nonces.py",
line 38, in get_repo_free_nonce return self.repository.get_free_nonce()
File "/usr/local/lib/python3.9/dist-packages/borg/repository.py", line
352, in get_free_nonce return int.from_bytes(unhexlify(fd.read()),
byteorder='big') binascii.Error: Non-hexadecimal digit found

Platform: Linux msi 5.10.0-17-amd64 #1 SMP Debian 5.10.136-1
(2022-08-13) x86_64 Linux: Unknown Linux  
Borg: 1.2.2  Python: CPython 3.9.2 msgpack: 1.0.4 fuse: pyfuse3 3.2.0
[pyfuse3,llfuse] PID: 2825473  CWD: /root
sys.argv: ['/usr/local/bin/borg', 'create', '--verbose',
'--exclude-caches', '--show-rc', '--warning', '--filter', 'AME',
'--list', '--stats', '--checkpoint-interval', '600', '--comment', ' 1er
backup après RAZ (plantage repo à cause de compact)',
'--chunker-params', '16,23,16,4095', '--compression', 'auto,zlib,6',
'--exclude-from', '/usr/local/sbin/BORG_EXCLUSIONS.list',
'::{hostname}-{utcnow}Z', '/'] SSH_ORIGINAL_COMMAND: None

terminating with error status, rc 2


My former test was a bit more successful (with msgpack from debian
package), I saw files beginning to pile into data/0/nnnn but it failed
after a while with these messages :

[…]
A /home/niff/Images/0_TMP/.thumbnails/r_899509.jpg.png
A /home/niff/Images/0_TMP/.thumbnails/r_899510.jpg.png
Repository hints file missing or corrupted, trying to recover:
unpack(b) received extra data.
Local Exception
Traceback (most recent call last):
  File
"/usr/local/lib/python3.9/dist-packages/borg/helpers/msgpack.py", line 169, in unpack
    return mp_unpack(stream, **kw)
  File "/usr/lib/python3/dist-packages/msgpack/__init__.py", line 46,
in unpack
    return unpackb(data, **kwargs)
  File "msgpack/_unpacker.pyx", line 202, in msgpack._cmsgpack.unpackb
msgpack.exceptions.ExtraData: unpack(b) received extra data.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/borg/repository.py",
line 576, in prepare_txn
    hints = msgpack.unpack(fd)
  File
"/usr/local/lib/python3.9/dist-packages/borg/helpers/msgpack.py", line 171, in unpack
    raise UnpackException(e)
borg.helpers.msgpack.UnpackException: unpack(b) received extra data.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File
"/usr/local/lib/python3.9/dist-packages/borg/helpers/msgpack.py", line 169, in unpack
    return mp_unpack(stream, **kw)
  File "/usr/lib/python3/dist-packages/msgpack/__init__.py", line 46,
in unpack
    return unpackb(data, **kwargs)
  File "msgpack/_unpacker.pyx", line 202, in msgpack._cmsgpack.unpackb
msgpack.exceptions.ExtraData: unpack(b) received extra data.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
5159, in main
    exit_code = archiver.run(args)
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
5090, in run
    return set_ec(func(args))
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
183, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
675, in do_create
    create_inner(archive, cache, fso)
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
608, in create_inner
    self._rec_walk(path=path, parent_fd=parent_fd, name=name,
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
820, in _rec_walk
    self._rec_walk(
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
820, in _rec_walk
    self._rec_walk(
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
820, in _rec_walk
    self._rec_walk(
  [Previous line repeated 3 more times]
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
786, in _rec_walk
    status = self._process_any(path=path, parent_fd=parent_fd,
name=name, st=st, fso=fso, cache=cache,
  File "/usr/local/lib/python3.9/dist-packages/borg/archiver.py", line
688, in _process_any
    return fso.process_file(path=path, parent_fd=parent_fd, name=name,
st=st, cache=cache)
  File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
1434, in process_file
    self.process_file_chunks(item, cache, self.stats,
self.show_progress, backup_io_iter(self.chunker.chunkify(None, fd)))
  File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
1264, in process_file_chunks
    from_chunk, part_number = self.maybe_checkpoint(item, from_chunk,
part_number, forced=False)
  File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
1238, in maybe_checkpoint
    from_chunk, part_number = self.write_part_file(item, from_chunk,
part_number)
  File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
1229, in write_part_file
    self.write_checkpoint()
  File "/usr/local/lib/python3.9/dist-packages/borg/archive.py", line
605, in write_checkpoint
    self.cache.chunk_decref(self.id, self.stats)
  File "/usr/local/lib/python3.9/dist-packages/borg/cache.py", line
978, in chunk_decref
    self.repository.delete(id, wait=wait)
  File "/usr/local/lib/python3.9/dist-packages/borg/repository.py",
line 1246, in delete
    self.prepare_txn(self.get_transaction_id())
  File "/usr/local/lib/python3.9/dist-packages/borg/repository.py",
line 584, in prepare_txn
    self.prepare_txn(transaction_id)
  File "/usr/local/lib/python3.9/dist-packages/borg/repository.py",
line 558, in prepare_txn
    self.index = self.open_index(transaction_id, auto_recover=False)
  File "/usr/local/lib/python3.9/dist-packages/borg/repository.py",
line 523, in open_index
    integrity_data = self._read_integrity(transaction_id, b'index')
  File "/usr/local/lib/python3.9/dist-packages/borg/repository.py",
line 511, in _read_integrity
    integrity = msgpack.unpack(fd)
  File
"/usr/local/lib/python3.9/dist-packages/borg/helpers/msgpack.py", line 171, in unpack
    raise UnpackException(e)
borg.helpers.msgpack.UnpackException: unpack(b) received extra data.

Platform: Linux msi 5.10.0-17-amd64 #1 SMP Debian 5.10.136-1
(2022-08-13) x86_64
Linux: Unknown Linux  
Borg: 1.2.2  Python: CPython 3.9.2 msgpack: 1.0.0 fuse: pyfuse3 3.2.0
[pyfuse3,llfuse]
PID: 2808327  CWD: /root
sys.argv: ['/usr/local/bin/borg', 'create', '--verbose',
'--exclude-caches', '--show-rc', '--warning', '--filter', 'AME', '--list', '--stats', '--checkpoint-interval', '600', '--comment', ' 1er backup après RAZ (plantage repo à cause de compact)', '--chunker-params', '16,23,16,4095', '--compression', 'auto,zlib,6', '--exclude-from', '/usr/local/sbin/BORG_EXCLUSIONS.list', '::{hostname}-{utcnow}Z', '/']
SSH_ORIGINAL_COMMAND: None

terminating with error status, rc 2


This a real concern, as a compaction test seems to have had clobbered
all of my repos :( so I really need to be able to make fresh backups
ASAP.

What is going wrong ?

Jean-Yves

More information about the Borgbackup mailing list