[Borgbackup] Consistent unexpectedly large deduplicated size

[Ed Blackman]

I have a couple of backups going to the same repo.  They are in the same repo because files sometimes move between backup sets.  I have different prefixes for the backups so that I can have different prune policies.

One of them has a consistent unexpectedly large deduplicated size, much larger than the sum size of the new files being added.

Local and server borg are version 1.2.4 as packaged by Debian 12.

Common command line options to both backups:
 /usr/bin/borg create --chunker-params 19,23,21,4095 --stats --list
 --show-rc --keep-exclude-tags --exclude-caches --exclude-if-present
 .nobackup --noatime

Backup A (the one with the unexpected size) had 45,364,409 bytes in 300 new files.  Options specific to this backup were:
 --compression=lz4
 --patterns-from=/home/backups/borg/snapshot/borg-tmp.20240426.1iExpB/hometmp-patterns
 server:/path::a_backup=host at 2024-04-26T04:18:02 home

Stats output for Backup A included:

Number of files: 316114
                      Original size      Compressed size    Deduplicated size
This archive:               32.14 GB             28.46 GB            215.98 MB
All archives:               43.55 TB             39.14 TB            667.32 GB

                      Unique chunks         Total chunks
Chunk index:                 1069818             73111633

Backup B had 4,226,032 bytes in 10 new files.  Options specific to this backup were:
 --compression=zlib --exclude-from=/etc/backup/borg.all.exclude
 server:/path::b_backup=host at 2024-04-26T04:18:02 home/user/projects

Stats output for Backup B included:
Number of files: 79413
                      Original size      Compressed size    Deduplicated size
This archive:                6.10 GB              5.08 GB              6.02 kB
All archives:               43.52 TB             39.12 TB            667.14 GB

                      Unique chunks         Total chunks
Chunk index:                 1069619             72877206


I'm not sure why backup A's deduplicated size is so much larger than the size of the new files in it.  Are there any ideas about how to find out?

-- 
Ed Blackman