[Borgbackup] first backup of large amounts

[Boris Kirkorowicz]

Hello,

Am 26.01.23 um 16:19 schrieb Thorsten Schöning:
> Guten Tag Boris Kirkorowicz,
> am Donnerstag, 26. Januar 2023 um 15:34 schrieben Sie:
> 
>> I want to backup a somewhat larger amount of data, approximately 35
>> TB. My first tries let me assume that 70 GB need about 1 hour to
>> backup. These 35 TB would take about 2~3 weeks in the given
>> environment. That means that the server would not be available
>> during an unacceptable long time.
> 
> That needs to be explained mor detailed: What kind of data and or
> use-case with that data makes you believe that you need to shut
> services down?

The server shall be taken offline during backup 1. to prevent files 
changed, 2. for security reasons.


> Concepts like snapshots in file systems like BTRFS and
> ZFS or volume managers like LVM are used to keep files consistent even
> for a longer period of time.

Target file system is ext4, no snapshots so far.


>> So I wonder if it would be
>> possible to split the backup into pieces of max. 400 GB (~ 6 hours)
>> while getting a single repository at the end.
> 
> That use-case is supported by creating checkpoint files as often as
> you like. Though, whenever backups are started, a complete directory
> listing will be done, which might take some additional overhead time
> for your. Things heavily depend on the number of files, their average
> size and stuff.
> 
>> -c SECONDS, --checkpoint-interval SECONDS       write checkpoint
>> every SECONDS seconds (Default: 1800)
> 
> https://borgbackup.readthedocs.io/en/stable/usage/create.html
> https://borgbackup.readthedocs.io/en/stable/faq.html#if-a-backup-stops-mid-way-does-the-already-backed-up-data-stay-there

Thx. As far as I understand, these checkpoints are created by default, 
and setting intervals is optional to adapt it to individual preferences 
-right? So if borg --create is stopped (killed?) during backing up 
files, the next time it is invoked it just checks hashes up to the last 
checkpoint, what is very fast, and then continues the normal way until 
it ends or is stopped again. Thus, I could simply start borg --create at 
night, stop it after 6 hours, and repeat this every night until it got 
all files (return code = 0).

Did I get it right?


-- 
Mit freundlichem Gruß                                 Best regards
                                  Kirkorowicz