[Borgbackup] Possible Bug?

[Martin Hoßbach]

Dear community,

I've run into a problem creating a backup. Borg 1.1.8 crashes with this
error report when creating a certain archive:


Processing files ...


Cleaned up 0 uncommitted segment files (== everything after segment
614750).

Local Exception
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 4436, in
main
    exit_code = archiver.run(args)
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 4368, in
run
    return set_ec(func(args))
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 152, in
wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 555, in
do_create
    create_inner(archive, cache)
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 518, in
create_inner
    read_special=args.read_special, dry_run=dry_run, st=st)
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 632, in
_process
    read_special=read_special, dry_run=dry_run)
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 632, in
_process
    read_special=read_special, dry_run=dry_run)
  File "/usr/lib/python3.7/site-packages/borg/archiver.py", line 606, in
_process
    status = archive.process_file(path, st, cache)
  File "/usr/lib/python3.7/site-packages/borg/archive.py", line 1028, in
process_file
    self.chunk_file(item, cache, self.stats,
backup_io_iter(self.chunker.chunkify(fd, fh)))
  File "/usr/lib/python3.7/site-packages/borg/archive.py", line 956, in
chunk_file
    item.chunks.append(chunk_processor(data))
  File "/usr/lib/python3.7/site-packages/borg/archive.py", line 944, in
chunk_processor
    chunk_entry = cache.add_chunk(self.key.id_hash(data), data, stats,
wait=False)
  File "/usr/lib/python3.7/site-packages/borg/cache.py", line 898, in
add_chunk
    self.repository.put(id, data, wait=wait)
  File "/usr/lib/python3.7/site-packages/borg/repository.py", line 1076, in
put
    self.prepare_txn(self.get_transaction_id())
  File "/usr/lib/python3.7/site-packages/borg/repository.py", line 504, in
prepare_txn
    hints = msgpack.unpack(fd)
  File "/usr/lib/python3.7/site-packages/msgpack/__init__.py", line 57, in
unpack
    return unpackb(data, **kwargs)
  File "msgpack/_unpacker.pyx", line 187, in msgpack._cmsgpack.unpackb
ValueError: 338108 exceeds max_map_len(32768)

Platform: Linux vault82 4.20.1-arch1-1-ARCH #1 SMP PREEMPT Wed Jan 9
20:25:43 UTC 2019 x86_64
Linux: arch
Borg: 1.1.8  Python: CPython 3.7.2
PID: 5997  CWD: /mnt/backup/clients/mirror/root
sys.argv: ['/usr/bin/borg', 'create', '--debug', '-p', '-s', '-C', 'zlib',
'--numeric-owner', '-x', '-e', 're:/swapfile1', '-e', 're:/video-tablet/',
'-e', 're:/pagefile.sys', '-e', 're:/hiberfil.sys', '-e',
're:/swapfile.sys', '--exclude-caches', '--keep-exclude-tags',
'::mirror-root-{now:%s}', '.']
SSH_ORIGINAL_COMMAND: None


The same backup runs smoothly when I use a newly-created repository, and as
well when I install the git version of borg (1.2.0.dev737+g4d29747f). Also,
borg check finds no problems in my repository.

So, I could now simply wait until borg 1.2.0 is released, but I could also
help to track that issue in borg. However, I'm not really fluent in Python,
so I could use some pointers what the developers would prefer, and where to
start?

Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/borgbackup/attachments/20190117/103f14e9/attachment.html>